idnits 2.17.1 draft-dulaunoy-misp-core-format-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** There are 11 instances of too long lines in the document, the longest one being 18 characters in excess of 72. ** The abstract seems to contain references ([MISP-P]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (April 10, 2018) is 2209 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Unused Reference: 'MISP-R' is defined on line 2050, but no explicit reference was found in the text == Unused Reference: 'MISP-T' is defined on line 2054, but no explicit reference was found in the text ** Obsolete normative reference: RFC 4627 (Obsoleted by RFC 7158, RFC 7159) Summary: 4 errors (**), 0 flaws (~~), 4 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Dulaunoy 3 Internet-Draft A. Iklody 4 Intended status: Informational CIRCL 5 Expires: October 12, 2018 April 10, 2018 7 MISP core format 8 draft-dulaunoy-misp-core-format-04 10 Abstract 12 This document describes the MISP core format used to exchange 13 indicators and threat information between MISP (Malware Information 14 and threat Sharing Platform) instances. The JSON format includes the 15 overall structure along with the semantic associated for each 16 respective key. The format is described to support other 17 implementations which reuse the format and ensuring an 18 interoperability with existing MISP [MISP-P] software and other 19 Threat Intelligence Platforms. 21 Status of This Memo 23 This Internet-Draft is submitted in full conformance with the 24 provisions of BCP 78 and BCP 79. 26 Internet-Drafts are working documents of the Internet Engineering 27 Task Force (IETF). Note that other groups may also distribute 28 working documents as Internet-Drafts. The list of current Internet- 29 Drafts is at https://datatracker.ietf.org/drafts/current/. 31 Internet-Drafts are draft documents valid for a maximum of six months 32 and may be updated, replaced, or obsoleted by other documents at any 33 time. It is inappropriate to use Internet-Drafts as reference 34 material or to cite them other than as "work in progress." 36 This Internet-Draft will expire on October 12, 2018. 38 Copyright Notice 40 Copyright (c) 2018 IETF Trust and the persons identified as the 41 document authors. All rights reserved. 43 This document is subject to BCP 78 and the IETF Trust's Legal 44 Provisions Relating to IETF Documents 45 (https://trustee.ietf.org/license-info) in effect on the date of 46 publication of this document. Please review these documents 47 carefully, as they describe your rights and restrictions with respect 48 to this document. Code Components extracted from this document must 49 include Simplified BSD License text as described in Section 4.e of 50 the Trust Legal Provisions and are provided without warranty as 51 described in the Simplified BSD License. 53 Table of Contents 55 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 56 1.1. Conventions and Terminology . . . . . . . . . . . . . . . 3 57 2. Format . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 58 2.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . 3 59 2.2. Event . . . . . . . . . . . . . . . . . . . . . . . . . . 3 60 2.2.1. Event Attributes . . . . . . . . . . . . . . . . . . 3 61 2.3. Objects . . . . . . . . . . . . . . . . . . . . . . . . . 7 62 2.3.1. Org . . . . . . . . . . . . . . . . . . . . . . . . . 7 63 2.3.2. Orgc . . . . . . . . . . . . . . . . . . . . . . . . 8 64 2.4. Attribute . . . . . . . . . . . . . . . . . . . . . . . . 8 65 2.4.1. Sample Attribute Object . . . . . . . . . . . . . . . 8 66 2.4.2. Attribute Attributes . . . . . . . . . . . . . . . . 9 67 2.5. ShadowAttribute . . . . . . . . . . . . . . . . . . . . . 14 68 2.5.1. Sample Attribute Object . . . . . . . . . . . . . . . 15 69 2.5.2. ShadowAttribute Attributes . . . . . . . . . . . . . 15 70 2.5.3. Org . . . . . . . . . . . . . . . . . . . . . . . . . 20 71 2.6. Object . . . . . . . . . . . . . . . . . . . . . . . . . 21 72 2.6.1. Sample Object object . . . . . . . . . . . . . . . . 21 73 2.6.2. Object Attributes . . . . . . . . . . . . . . . . . . 22 74 2.7. Object References . . . . . . . . . . . . . . . . . . . . 25 75 2.7.1. Sample ObjectReference object . . . . . . . . . . . . 25 76 2.7.2. ObjectReference Attributes . . . . . . . . . . . . . 26 77 2.8. Tag . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 78 2.8.1. Sample Tag . . . . . . . . . . . . . . . . . . . . . 28 79 2.9. Sighting . . . . . . . . . . . . . . . . . . . . . . . . 28 80 2.9.1. Sample Sighting . . . . . . . . . . . . . . . . . . . 30 81 2.10. Galaxy . . . . . . . . . . . . . . . . . . . . . . . . . 30 82 2.10.1. Sample Galaxy . . . . . . . . . . . . . . . . . . . 30 83 3. JSON Schema . . . . . . . . . . . . . . . . . . . . . . . . . 32 84 4. Manifest . . . . . . . . . . . . . . . . . . . . . . . . . . 41 85 4.1. Format . . . . . . . . . . . . . . . . . . . . . . . . . 41 86 4.1.1. Sample Manifest . . . . . . . . . . . . . . . . . . . 42 87 5. Implementation . . . . . . . . . . . . . . . . . . . . . . . 43 88 6. Security Considerations . . . . . . . . . . . . . . . . . . . 43 89 7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 44 90 8. Sample MISP file . . . . . . . . . . . . . . . . . . . . . . 44 91 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 44 92 9.1. Normative References . . . . . . . . . . . . . . . . . . 44 93 9.2. Informative References . . . . . . . . . . . . . . . . . 44 94 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 45 96 1. Introduction 98 Sharing threat information became a fundamental requirements in the 99 Internet, security and intelligence community at large. Threat 100 information can include indicators of compromise, malicious file 101 indicators, financial fraud indicators or even detailed information 102 about a threat actor. MISP [MISP-P] started as an open source 103 project in late 2011 and the MISP format started to be widely used as 104 an exchange format within the community in the past years. The aim 105 of this document is to describe the specification and the MISP core 106 format. 108 1.1. Conventions and Terminology 110 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 111 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 112 document are to be interpreted as described in RFC 2119 [RFC2119]. 114 2. Format 116 2.1. Overview 118 The MISP core format is in the JSON [RFC4627] format. In MISP, an 119 event is composed of a single JSON object. 121 A capitalized key (like Event, Org) represent a data model and a non- 122 capitalised key is just an attribute. This nomenclature can support 123 an implementation to represent the MISP format in another data 124 structure. 126 2.2. Event 128 An event is a simple meta structure scheme where attributes and meta- 129 data are embedded to compose a coherent set of indicators. An event 130 can be composed from an incident, a security analysis report or a 131 specific threat actor analysis. The meaning of an event only depends 132 of the information embedded in the event. 134 2.2.1. Event Attributes 136 2.2.1.1. uuid 138 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 139 the event. The uuid MUST be preserved for any updates or transfer of 140 the same event. UUID version 4 is RECOMMENDED when assigning it to a 141 new event. 143 uuid is represented as a JSON string. uuid MUST be present. 145 2.2.1.2. id 147 id represents the human-readable identifier associated to the event 148 for a specific MISP instance. A human-readable identifier MUST be 149 represented as an unsigned integer. 151 id is represented as a JSON string. id SHALL be present. 153 2.2.1.3. published 155 published represents the event publication state. If the event was 156 published, the published value MUST be true. In any other 157 publication state, the published value MUST be false. 159 published is represented as a JSON boolean. published MUST be 160 present. 162 2.2.1.4. info 164 info represents the information field of the event. info is a free- 165 text value to provide a human-readable summary of the event. info 166 SHOULD NOT be bigger than 256 characters and SHOULD NOT include new- 167 lines. 169 info is represented as a JSON string. info MUST be present. 171 2.2.1.5. threat_level_id 173 threat_level_id represents the threat level. 175 4: 176 Undefined 178 3: 179 Low 181 2: 182 Medium 184 1: 185 High 187 If a higher granularity is required, a MISP taxonomy applied as a Tag 188 SHOULD be preferred. 190 threat_level_id is represented as a JSON string. threat_level_id 191 SHALL be present. 193 2.2.1.6. analysis 195 analysis represents the analysis level. 197 0: 198 Initial 200 1: 201 Ongoing 203 2: 204 Complete 206 If a higher granularity is required, a MISP taxonomy applied as a Tag 207 SHOULD be preferred. 209 analysis is represented as a JSON string. analysis SHALL be present. 211 2.2.1.7. date 213 date represents a reference date to the event in ISO 8601 format 214 (date only: YYYY-MM-DD). This date corresponds to the date the event 215 occurred, which may be in the past. 217 date is represented as a JSON string. date MUST be present. 219 2.2.1.8. timestamp 221 timestamp represents a reference time when the event, or one of the 222 attributes within the event was created, or last updated/edited on 223 the instance. timestamp is expressed in seconds (decimal) since 1st 224 of January 1970 (Unix timestamp). The time zone MUST be UTC. 226 timestamp is represented as a JSON string. timestamp MUST be present. 228 2.2.1.9. publish_timestamp 230 publish_timestamp represents a reference time when the event was 231 published on the instance. published_timestamp is expressed in 232 seconds (decimal) since 1st of January 1970 (Unix timestamp). At 233 each publication of an event, publish_timestamp MUST be updated. The 234 time zone MUST be UTC. If the published_timestamp is present and the 235 published flag is set to false, the publish_timestamp represents the 236 previous publication timestamp. If the event was never published, 237 the published_timestamp MUST be set to 0. 239 publish_timestamp is represented as a JSON string. publish_timestamp 240 MUST be present. 242 2.2.1.10. org_id 244 org_id represents a human-readable identifier referencing an Org 245 object of the organisation which generated the event. A human- 246 readable identifier MUST be represented as an unsigned integer. 248 The org_id MUST be updated when the event is generated by a new 249 instance. 251 org_id is represented as a JSON string. org_id MUST be present. 253 2.2.1.11. orgc_id 255 orgc_id represents a human-readable identifier referencing an Orgc 256 object of the organisation which created the event. 258 The orgc_id and Org object MUST be preserved for any updates or 259 transfer of the same event. 261 orgc_id is represented as a JSON string. orgc_id MUST be present. 263 2.2.1.12. attribute_count 265 attribute_count represents the number of attributes in the event. 266 attribute_count is expressed in decimal. 268 attribute_count is represented as a JSON string. attribute_count 269 SHALL be present. 271 2.2.1.13. distribution 273 distribution represents the basic distribution rules of the event. 274 The system must adhere to the distribution setting for access control 275 and for dissemination of the event. 277 distribution is represented by a JSON string. distribution MUST be 278 present and be one of the following options: 280 0 281 Your Organisation Only 283 1 284 This Community Only 286 2 287 Connected Communities 289 3 290 All Communities 292 4 293 Sharing Group 295 2.2.1.14. sharing_group_id 297 sharing_group_id represents a human-readable identifier referencing a 298 Sharing Group object that defines the distribution of the event, if 299 distribution level "4" is set. A human-readable identifier MUST be 300 represented as an unsigned integer. 302 sharing_group_id is represented by a JSON string and SHOULD be 303 present. If a distribution level other than "4" is chosen the 304 sharing_group_id MUST be set to "0". 306 2.2.1.15. extends_uuid 308 extends_uuid represents which event is extended by this event. The 309 extends_uuid is described as a Universally Unique IDentifier (UUID) 310 [RFC4122] with the UUID of the extended event. 312 extends_uuid is represented as a JSON string. extends_uuid SHOULD be 313 present. 315 2.3. Objects 317 2.3.1. Org 319 An Org object is composed of an uuid, name and id. 321 The uuid represents the Universally Unique IDentifier (UUID) 322 [RFC4122] of the organisation. The organisation UUID is globally 323 assigned to an organisation and SHALL be kept overtime. 325 The name is a readable description of the organisation and SHOULD be 326 present. The id is a human-readable identifier generated by the 327 instance and used as reference in the event. A human-readable 328 identifier MUST be represented as an unsigned integer. 330 uuid, name and id are represented as a JSON string. uuid, name and id 331 MUST be present. 333 2.3.1.1. Sample Org Object 334 "Org": { 335 "id": "2", 336 "name": "CIRCL", 337 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f" 338 } 340 2.3.2. Orgc 342 An Orgc object is composed of an uuid, name and id. 344 The uuid MUST be preserved for any updates or transfer of the same 345 event. UUID version 4 is RECOMMENDED when assigning it to a new 346 event. The organisation UUID is globally assigned to an organisation 347 and SHALL be kept overtime. 349 The name is a readable description of the organisation and SHOULD be 350 present. The id is a human-readable identifier generated by the 351 instance and used as reference in the event. A human-readable 352 identifier MUST be represented as an unsigned integer. 354 uuid, name and id are represented as a JSON string. uuid, name and id 355 MUST be present. 357 2.4. Attribute 359 Attributes are used to describe the indicators and contextual data of 360 an event. The main information contained in an attribute is made up 361 of a category-type-value triplet, where the category and type give 362 meaning and context to the value. Through the various category-type 363 combinations a wide range of information can be conveyed. 365 A MISP document MUST at least includes category-type-value triplet 366 described in section "Attribute Attributes". 368 2.4.1. Sample Attribute Object 369 "Attribute": { 370 "id": "346056", 371 "type": "comment", 372 "category": "Other", 373 "to_ids": false, 374 "uuid": "57f4f6d9-cd20-458b-84fd-109ec0a83869", 375 "event_id": "3357", 376 "distribution": "5", 377 "timestamp": "1475679332", 378 "comment": "", 379 "sharing_group_id": "0", 380 "deleted": false, 381 "value": "Hello world", 382 "SharingGroup": [], 383 "ShadowAttribute": [], 384 "RelatedAttribute": [] 385 } 387 2.4.2. Attribute Attributes 389 2.4.2.1. uuid 391 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 392 the event. The uuid MUST be preserved for any updates or transfer of 393 the same event. UUID version 4 is RECOMMENDED when assigning it to a 394 new event. 396 uuid is represented as a JSON string. uuid MUST be present. 398 2.4.2.2. id 400 id represents the human-readable identifier associated to the event 401 for a specific MISP instance. A human-readable identifier MUST be 402 represented as an unsigned integer. 404 id is represented as a JSON string. id SHALL be present. 406 2.4.2.3. type 408 type represents the means through which an attribute tries to 409 describe the intent of the attribute creator, using a list of pre- 410 defined attribute types. 412 type is represented as a JSON string. type MUST be present and it 413 MUST be a valid selection for the chosen category. The list of valid 414 category-type combinations is as follows: 416 Internal reference 417 text, link, comment, other, hex 419 Targeting data 420 target-user, target-email, target-machine, target-org, target- 421 location, target-external, comment 423 Antivirus detection 424 link, comment, text, hex, attachment, other 426 Payload delivery 427 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 428 ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, filename, 429 filename|md5, filename|sha1, filename|sha224, filename|sha256, 430 filename|sha384, filename|sha512, filename|sha512/224, 431 filename|sha512/256, filename|authentihash, filename|ssdeep, 432 filename|tlsh, filename|imphash, filename|impfuzzy, 433 filename|pehash, ip-src, ip-dst, hostname, domain, email-src, 434 email-dst, email-subject, email-attachment, url, user-agent, AS, 435 pattern-in-file, pattern-in-traffic, yara, attachment, malware- 436 sample, link, malware-type, mime-type, comment, text, 437 vulnerability, x509-fingerprint-sha1, other, ip-dst|port, ip- 438 src|port, hostname|port, email-dst-display-name, email-src- 439 display-name, email-header, email-reply-to, email-x-mailer, email- 440 mime-boundary, email-thread-index, email-message-id, mobile- 441 application-id 443 Artifacts dropped 444 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 445 ssdeep, imphash, impfuzzy, authentihash, filename, filename|md5, 446 filename|sha1, filename|sha224, filename|sha256, filename|sha384, 447 filename|sha512, filename|sha512/224, filename|sha512/256, 448 filename|authentihash, filename|ssdeep, filename|tlsh, 449 filename|imphash, filename|impfuzzy, filename|pehash, regkey, 450 regkey|value, pattern-in-file, pattern-in-memory, pdb, yara, 451 sigma, stix2-pattern, gene, attachment, malware-sample, mime-type, 452 named pipe, mutex, windows-scheduled-task, windows-service-name, 453 windows-service-displayname, comment, text, hex, x509-fingerprint- 454 sha1, other 456 Payload installation 457 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 458 ssdeep, imphash, authentihash, pehash, tlsh, filename, 459 filename|md5, filename|sha1, filename|sha224, filename|sha256, 460 filename|sha384, filename|sha512, filename|sha512/224, 461 filename|sha512/256, filename|authentihash, filename|ssdeep, 462 filename|tlsh, filename|imphash, filename|pehash, pattern-in-file, 463 mime-type, pattern-in-traffic, pattern-in-memory, yara, 464 stix2-pattern, vulnerability, attachment, malware-sample, malware- 465 type, comment, text, hex, x509-fingerprint-sha1, mobile- 466 application-id, other 468 Persistence mechanism 469 filename, regkey, regkey|value, comment, text, other, text 471 Network activity 472 ip-src, ip-dst, hostname, domain, domain|ip, email-dst, url, uri, 473 user-agent, http-method, AS, snort, pattern-in-file, pattern-in- 474 traffic, stix2-pattern, attachment, comment, text, x509- 475 fingerprint-sha1, other, hex, cookie 477 Payload type 478 comment, text, other 480 Attribution 481 threat-actor, campaign-name, campaign-id, whois-registrant-phone, 482 whois-registrant-email, whois-registrant-name, whois-registrar, 483 whois-creation-date, comment, text, x509-fingerprint-sha1, other 485 External analysis 486 md5, sha1, sha256, filename, filename|md5, filename|sha1, 487 filename|sha256, ip-src, ip-dst, hostname, domain, domain|ip, url, 488 user-agent, regkey, regkey|value, AS, snort, pattern-in-file, 489 pattern-in-traffic, pattern-in-memory, vulnerability, attachment, 490 malware-sample, link, comment, text, x509-fingerprint-sha1, 491 github-repository, other 493 Financial fraud 494 btc, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, 495 phone-number, comment, text, other, hex 497 Support tool 498 attachment, link, comment, text, other, hex 500 Social network 501 github-username, github-repository, github-organisation, jabber- 502 id, twitter-id, email-src, email-dst, comment, text, other 504 Person 505 first-name, middle-name, last-name, date-of-birth, place-of-birth, 506 gender, passport-number, passport-country, passport-expiration, 507 redress-number, nationality, visa-number, issue-date-of-the-visa, 508 primary-residence, country-of-residence, special-service-request, 509 frequent-flyer-number, travel-details, payment-details, place- 510 port-of-original-embarkation, place-port-of-clearance, place-port- 511 of-onward-foreign-destination, passenger-name-record-locator- 512 number, comment, text, other, phone-number, identity-card-number 514 Other 515 comment, text, other, size-in-bytes, counter, datetime, cpe, port, 516 float, hex, phone-number 518 Attributes are based on the usage within their different communities. 519 Attributes can be extended on a regular basis and this reference 520 document is updated accordingly. 522 2.4.2.4. category 524 category represents the intent of what the attribute is describing as 525 selected by the attribute creator, using a list of pre-defined 526 attribute categories. 528 category is represented as a JSON string. category MUST be present 529 and it MUST be a valid selection for the chosen type. The list of 530 valid category-type combinations is mentioned above. 532 2.4.2.5. to_ids 534 to_ids represents whether the attribute is meant to be actionable. 535 Actionable defined attributes that can be used in automated processes 536 as a pattern for detection in Local or Network Intrusion Detection 537 System, log analysis tools or even filtering mechanisms. 539 to_ids is represented as a JSON boolean. to_ids MUST be present. 541 2.4.2.6. event_id 543 event_id represents a human-readable identifier referencing the Event 544 object that the attribute belongs to. A human-readable identifier 545 MUST be represented as an unsigned integer. 547 The event_id SHOULD be updated when the event is imported to reflect 548 the newly created event's id on the instance. 550 event_id is represented as a JSON string. event_id MUST be present. 552 2.4.2.7. distribution 554 distribution represents the basic distribution rules of the 555 attribute. The system must adhere to the distribution setting for 556 access control and for dissemination of the attribute. 558 distribution is represented by a JSON string. distribution MUST be 559 present and be one of the following options: 561 0 562 Your Organisation Only 564 1 565 This Community Only 567 2 568 Connected Communities 570 3 571 All Communities 573 4 574 Sharing Group 576 5 577 Inherit Event 579 2.4.2.8. timestamp 581 timestamp represents a reference time when the attribute was created 582 or last modified. timestamp is expressed in seconds (decimal) since 583 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. 585 timestamp is represented as a JSON string. timestamp MUST be present. 587 2.4.2.9. comment 589 comment is a contextual comment field. 591 comment is represented by a JSON string. comment MAY be present. 593 2.4.2.10. sharing_group_id 595 sharing_group_id represents a human-readable identifier referencing a 596 Sharing Group object that defines the distribution of the attribute, 597 if distribution level "4" is set. A human-readable identifier MUST 598 be represented as an unsigned integer. 600 sharing_group_id is represented by a JSON string and SHOULD be 601 present. If a distribution level other than "4" is chosen the 602 sharing_group_id MUST be set to "0". 604 2.4.2.11. deleted 606 deleted represents a setting that allows attributes to be revoked. 607 Revoked attributes are not actionable and exist merely to inform 608 other instances of a revocation. 610 deleted is represented by a JSON boolean. deleted MUST be present. 612 2.4.2.12. data 614 data contains the base64 encoded contents of an attachment or a 615 malware sample. For malware samples, the sample MUST be encrypted 616 using a password protected zip archive, with the password being 617 "infected". 619 data is represented by a JSON string in base64 encoding. data MUST be 620 set for attributes of type malware-sample and attachment. 622 2.4.2.13. RelatedAttribute 624 RelatedAttribute is an array of attributes correlating with the 625 current attribute. Each element in the array represents an JSON 626 object which contains an Attribute dictionnary with the external 627 attributes who correlate. Each Attribute MUST include the id, 628 org_id, info and a value. Only the correlations found on the local 629 instance are shown in RelatedAttribute. 631 RelatedAttribute MAY be present. 633 2.4.2.14. ShadowAttribute 635 ShadowAttribute is an array of shadow attributes that serve as 636 proposals by third parties to alter the containing attribute. The 637 structure of a ShadowAttribute is similar to that of an Attribute, 638 which can be accepted or discarded by the event creator. If 639 accepted, the original attribute containing the shadow attribute is 640 removed and the shadow attribute is converted into an attribute. 642 Each shadow attribute that references an attribute MUST contain the 643 containing attribute's ID in the old_id field and the event's ID in 644 the event_id field. 646 2.4.2.15. value 648 value represents the payload of an attribute. The format of the 649 value is dependent on the type of the attribute. 651 value is represented by a JSON string. value MUST be present. 653 2.5. ShadowAttribute 655 ShadowAttributes are 3rd party created attributes that either propose 656 to add new information to an event or modify existing information. 657 They are not meant to be actionable until the event creator accepts 658 them - at which point they will be converted into attributes or 659 modify an existing attribute. 661 They are similar in structure to Attributes but additionally carry a 662 reference to the creator of the ShadowAttribute as well as a 663 revocation flag. 665 2.5.1. Sample Attribute Object 667 "ShadowAttribute": { 668 "id": "8", 669 "type": "ip-src", 670 "category": "Network activity", 671 "to_ids": false, 672 "uuid": "57d475f1-da78-4569-89de-1458c0a83869", 673 "event_uuid": "57d475e6-41c4-41ca-b450-145ec0a83869", 674 "event_id": "9", 675 "old_id": "319", 676 "comment": "", 677 "org_id": "1", 678 "proposal_to_delete": false, 679 "value": "5.5.5.5", 680 "deleted": false, 681 "Org": { 682 "id": "1", 683 "name": "MISP", 684 "uuid": "568cce5a-0c80-412b-8fdf-1ffac0a83869" 685 } 686 } 688 2.5.2. ShadowAttribute Attributes 690 2.5.2.1. uuid 692 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 693 the event. The uuid MUST be preserved for any updates or transfer of 694 the same event. UUID version 4 is RECOMMENDED when assigning it to a 695 new event. 697 uuid is represented as a JSON string. uuid MUST be present. 699 2.5.2.2. id 701 id represents the human-readable identifier associated to the event 702 for a specific MISP instance. human-readable identifier MUST be 703 represented as an unsigned integer. id is represented as a JSON 704 string. id SHALL be present. 706 2.5.2.3. type 708 type represents the means through which an attribute tries to 709 describe the intent of the attribute creator, using a list of pre- 710 defined attribute types. 712 type is represented as a JSON string. type MUST be present and it 713 MUST be a valid selection for the chosen category. The list of valid 714 category-type combinations is as follows: 716 Internal reference 717 text, link, comment, other, hex 719 Targeting data 720 target-user, target-email, target-machine, target-org, target- 721 location, target-external, comment 723 Antivirus detection 724 link, comment, text, hex, attachment, other 726 Payload delivery 727 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 728 ssdeep, imphash, impfuzzy, authentihash, pehash, tlsh, filename, 729 filename|md5, filename|sha1, filename|sha224, filename|sha256, 730 filename|sha384, filename|sha512, filename|sha512/224, 731 filename|sha512/256, filename|authentihash, filename|ssdeep, 732 filename|tlsh, filename|imphash, filename|impfuzzy, 733 filename|pehash, ip-src, ip-dst, hostname, domain, email-src, 734 email-dst, email-subject, email-attachment, url, user-agent, AS, 735 pattern-in-file, pattern-in-traffic, yara, attachment, malware- 736 sample, link, malware-type, mime-type, comment, text, 737 vulnerability, x509-fingerprint-sha1, other, ip-dst|port, ip- 738 src|port, hostname|port, email-dst-display-name, email-src- 739 display-name, email-header, email-reply-to, email-x-mailer, email- 740 mime-boundary, email-thread-index, email-message-id, mobile- 741 application-id 743 Artifacts dropped 744 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 745 ssdeep, imphash, impfuzzy, authentihash, filename, filename|md5, 746 filename|sha1, filename|sha224, filename|sha256, filename|sha384, 747 filename|sha512, filename|sha512/224, filename|sha512/256, 748 filename|authentihash, filename|ssdeep, filename|tlsh, 749 filename|imphash, filename|impfuzzy, filename|pehash, regkey, 750 regkey|value, pattern-in-file, pattern-in-memory, pdb, yara, 751 sigma, gene, stix2-pattern, attachment, malware-sample, mime-type, 752 named pipe, mutex, windows-scheduled-task, windows-service-name, 753 windows-service-displayname, comment, text, hex, x509-fingerprint- 754 sha1, other 756 Payload installation 757 md5, sha1, sha224, sha256, sha384, sha512, sha512/224, sha512/256, 758 ssdeep, imphash, authentihash, pehash, tlsh, filename, 759 filename|md5, filename|sha1, filename|sha224, filename|sha256, 760 filename|sha384, filename|sha512, filename|sha512/224, 761 filename|sha512/256, filename|authentihash, filename|ssdeep, 762 filename|tlsh, filename|imphash, filename|pehash, mime-type, 763 pattern-in-file, pattern-in-traffic, pattern-in-memory, yara, 764 stix2-pattern, vulnerability, attachment, malware-sample, malware- 765 type, comment, text, hex, x509-fingerprint-sha1, mobile- 766 application-id, other 768 Persistence mechanism 769 filename, regkey, regkey|value, comment, text, other, text 771 Network activity 772 ip-src, ip-dst, hostname, domain, domain|ip, email-dst, url, uri, 773 user-agent, http-method, AS, snort, pattern-in-file, pattern-in- 774 traffic, stix2-pattern, attachment, comment, text, x509- 775 fingerprint-sha1, other, hex, cookie 777 Payload type 778 comment, text, other 780 Attribution 781 threat-actor, campaign-name, campaign-id, whois-registrant-phone, 782 whois-registrant-email, whois-registrant-name, whois-registrant- 783 org, whois-registrar, whois-creation-date, comment, text, x509- 784 fingerprint-sha1, other 786 External analysis 787 md5, sha1, sha256, filename, filename|md5, filename|sha1, 788 filename|sha256, ip-src, ip-dst, hostname, domain, domain|ip, url, 789 user-agent, regkey, regkey|value, AS, snort, pattern-in-file, 790 pattern-in-traffic, pattern-in-memory, vulnerability, attachment, 791 malware-sample, link, comment, text, x509-fingerprint-sha1, 792 github-repository, other 794 Financial fraud 795 btc, iban, bic, bank-account-nr, aba-rtn, bin, cc-number, prtn, 796 phone-number, comment, text, other, hex 798 Support tool 799 attachment, link, comment, text, other, hex 801 Social network 802 github-username, github-repository, github-organisation, jabber- 803 id, twitter-id, email-src, email-dst, comment, text, other 805 Person 806 first-name, middle-name, last-name, date-of-birth, place-of-birth, 807 gender, passport-number, passport-country, passport-expiration, 808 redress-number, nationality, visa-number, issue-date-of-the-visa, 809 primary-residence, country-of-residence, special-service-request, 810 frequent-flyer-number, travel-details, payment-details, place- 811 port-of-original-embarkation, place-port-of-clearance, place-port- 812 of-onward-foreign-destination, passenger-name-record-locator- 813 number, comment, text, other, phone-number, identity-card-number 815 Other 816 comment, text, other, size-in-bytes, counter, datetime, cpe, port, 817 float, hex, phone-number 819 Attributes are based on the usage within their different communities. 820 Attributes can be extended on a regular basis and this reference 821 document is updated accordingly. 823 2.5.2.4. category 825 category represents the intent of what the attribute is describing as 826 selected by the attribute creator, using a list of pre-defined 827 attribute categories. 829 category is represented as a JSON string. category MUST be present 830 and it MUST be a valid selection for the chosen type. The list of 831 valid category-type combinations is mentioned above. 833 2.5.2.5. to_ids 835 to_ids represents whether the Attribute to be created if the 836 ShadowAttribute is accepted is meant to be actionable. Actionable 837 defined attributes that can be used in automated processes as a 838 pattern for detection in Local or Network Intrusion Detection System, 839 log analysis tools or even filtering mechanisms. 841 to_ids is represented as a JSON boolean. to_ids MUST be present. 843 2.5.2.6. event_id 845 event_id represents a human-readable identifier referencing the Event 846 object that the ShadowAttribute belongs to. 848 The event_id SHOULD be updated when the event is imported to reflect 849 the newly created event's id on the instance. 851 event_id is represented as a JSON string. event_id MUST be present. 853 2.5.2.7. old_id 855 old_id represents a human-readable identifier referencing the 856 Attribute object that the ShadowAttribute belongs to. A 857 ShadowAttribute can this way target an existing Attribute, implying 858 that it is a proposal to modify an existing Attribute, or 859 alternatively it can be a proposal to create a new Attribute for the 860 containing Event. 862 The old_id SHOULD be updated when the event is imported to reflect 863 the newly created Attribute's id on the instance. Alternatively, if 864 the ShadowAttribute proposes the creation of a new Attribute, it 865 should be set to 0. 867 old_id is represented as a JSON string. old_id MUST be present. 869 2.5.2.8. timestamp 871 timestamp represents a reference time when the attribute was created 872 or last modified. timestamp is expressed in seconds (decimal) since 873 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. 875 timestamp is represented as a JSON string. timestamp MUST be present. 877 2.5.2.9. comment 879 comment is a contextual comment field. 881 comment is represented by a JSON string. comment MAY be present. 883 2.5.2.10. org_id 885 org_id represents a human-readable identifier referencing the 886 proposal creator's Organisation object. A human-readable identifier 887 MUST be represented as an unsigned integer. 889 Whilst attributes can only be created by the event creator 890 organisation, shadow attributes can be created by third parties. 891 org_id tracks the creator organisation. 893 org_id is represented by a JSON string and MUST be present. 895 2.5.2.11. proposal_to_delete 897 proposal_to_delete is a boolean flag that sets whether the shadow 898 attribute proposes to alter an attribute, or whether it proposes to 899 remove it completely. 901 Accepting a shadow attribute with this flag set will remove the 902 target attribute. 904 proposal_to_delete is a JSON boolean and it MUST be present. If 905 proposal_to_delete is set to true, old_id MUST NOT be 0. 907 2.5.2.12. deleted 909 deleted represents a setting that allows shadow attributes to be 910 revoked. Revoked shadow attributes only serve to inform other 911 instances that the shadow attribute is no longer active. 913 deleted is represented by a JSON boolean. deleted SHOULD be present. 915 2.5.2.13. data 917 data contains the base64 encoded contents of an attachment or a 918 malware sample. For malware samples, the sample MUST be encrypted 919 using a password protected zip archive, with the password being 920 "infected". 922 data is represented by a JSON string in base64 encoding. data MUST be 923 set for shadow attributes of type malware-sample and attachment. 925 2.5.3. Org 927 An Org object is composed of an uuid, name and id. 929 The uuid represents the Universally Unique IDentifier (UUID) 930 [RFC4122] of the organization. The organization UUID is globally 931 assigned to an organization and SHALL be kept overtime. 933 The name is a readable description of the organization and SHOULD be 934 present. The id is a human-readable identifier generated by the 935 instance and used as reference in the event. A human-readable 936 identifier MUST be represented as an unsigned integer. 938 uuid, name and id are represented as a JSON string. uuid, name and id 939 MUST be present. 941 2.5.3.1. Sample Org Object 943 "Org": { 944 "id": "2", 945 "name": "CIRCL", 946 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f" 947 } 949 2.5.3.2. value 951 value represents the payload of an attribute. The format of the 952 value is dependent on the type of the attribute. 954 value is represented by a JSON string. value MUST be present. 956 2.6. Object 958 Objects serve as a contextual bond between a list of attributes 959 within an event. Their main purpose is to describe more complex 960 structures than can be described by a single attribute Each object is 961 created using an Object Template and carries the meta-data of the 962 template used for its creation within. Objects belong to a meta- 963 category and are defined by a name. 965 The schema used is described by the template_uuid and 966 template_version fields. 968 A MISP document containing an Object MUST contain a name, a meta- 969 category, a description, a template_uuid and a template_version as 970 described in the "Object Attributes" section. 972 2.6.1. Sample Object object 973 "Object": { 974 "id": "588", 975 "name": "file", 976 "meta-category": "file", 977 "description": "File object describing a file with meta-information", 978 "template_uuid": "688c46fb-5edb-40a3-8273-1af7923e2215", 979 "template_version": "3", 980 "event_id": "56", 981 "uuid": "398b0094-0384-4c48-9bf0-22b3dff9c4d3", 982 "timestamp": "1505747965", 983 "distribution": "5", 984 "sharing_group_id": "0", 985 "comment": "", 986 "deleted": false, 987 "ObjectReference": [], 988 "Attribute": [ 989 "id": "7822", 990 "type": "filename", 991 "category": "Payload delivery", 992 "to_ids": true, 993 "uuid": "59bfe3fb-bde0-4dfe-b5b1-2b10a07724d1", 994 "event_id": "56", 995 "distribution": "0", 996 "timestamp": "1505747963", 997 "comment": "", 998 "sharing_group_id": "0", 999 "deleted": false, 1000 "disable_correlation": false, 1001 "object_id": "588", 1002 "object_relation": "filename", 1003 "value": "StarCraft.exe", 1004 "ShadowAttribute": [] 1005 ] 1006 } 1008 2.6.2. Object Attributes 1010 2.6.2.1. uuid 1012 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 1013 the object. The uuid MUST be preserved for any updates or transfer 1014 of the same object. UUID version 4 is RECOMMENDED when assigning it 1015 to a new object. 1017 2.6.2.2. id 1019 id represents the human-readable identifier associated to the object 1020 for a specific MISP instance. A human-readable identifier MUST be 1021 represented as an unsigned integer. 1023 id is represented as a JSON string. id SHALL be present. 1025 2.6.2.3. name 1027 name represents the human-readable name of the object describing the 1028 intent of the object package. 1030 name is represented as a JSON string. name MUST be present 1032 2.6.2.4. meta-category 1034 meta-category represents the sub-category of objects that the given 1035 object belongs to. meta-categories are not tied to a fixed list of 1036 options but can be created on the fly. 1038 meta-category is represented as a JSON string. meta-category MUST be 1039 present 1041 2.6.2.5. description 1043 description is a human-readable description of the given object type, 1044 as derived from the template used for creation. 1046 description is represented as a JSON string. id SHALL be present. 1048 2.6.2.6. template_uuid 1050 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 1051 the template used to create the object. The uuid MUST be preserved 1052 to preserve the object's association with the correct template used 1053 for creation. UUID version 4 is RECOMMENDED when assigning it to a 1054 new object. 1056 2.6.2.7. template_version 1058 template_version represents a numeric incrementing version of the 1059 template used to create the object. It is used to associate the 1060 object to the correct version of the template and together with the 1061 template_uuid forms an association to the correct template type and 1062 version. 1064 version is represented as a JSON string. version MUST be present. 1066 2.6.2.8. event_id 1068 event_id represents the human-readable identifier of the event that 1069 the object belongs to on a specific MISP instance. A human-readable 1070 identifier MUST be represented as an unsigned integer. 1072 event_id is represented as a JSON string. event_id SHALL be present. 1074 2.6.2.9. timestamp 1076 timestamp represents a reference time when the object was created or 1077 last modified. timestamp is expressed in seconds (decimal) since 1st 1078 of January 1970 (Unix timestamp). The time zone MUST be UTC. 1080 timestamp is represented as a JSON string. timestamp MUST be present. 1082 2.6.2.10. distribution 1084 distribution represents the basic distribution rules of the object. 1085 The system must adhere to the distribution setting for access control 1086 and for dissemination of the object. 1088 distribution is represented by a JSON string. distribution MUST be 1089 present and be one of the following options: 1091 0 1092 Your Organisation Only 1094 1 1095 This Community Only 1097 2 1098 Connected Communities 1100 3 1101 All Communities 1103 4 1104 Sharing Group 1106 2.6.2.11. sharing_group_id 1108 sharing_group_id represents a human-readable identifier referencing a 1109 Sharing Group object that defines the distribution of the object, if 1110 distribution level "4" is set. A human-readable identifier MUST be 1111 represented as an unsigned integer. 1113 sharing_group_id is represented by a JSON string and SHOULD be 1114 present. If a distribution level other than "4" is chosen the 1115 sharing_group_id MUST be set to "0". 1117 2.6.2.12. comment 1119 comment is a contextual comment field. 1121 comment is represented by a JSON string. comment MAY be present. 1123 2.6.2.13. deleted 1125 deleted represents a setting that allows attributes to be revoked. 1126 Revoked attributes are not actionable and exist merely to inform 1127 other instances of a revocation. 1129 deleted is represented by a JSON boolean. deleted MUST be present. 1131 2.6.2.14. Attribute 1133 Attribute is an array of attributes that describe the object with 1134 data. 1136 Each attribute in an object MUST contain the parent event's ID in the 1137 event_id field and the parent object's ID in the object_id field. 1139 2.7. Object References 1141 Object References serve as a logical link between an Object and 1142 another referenced Object or Attribute. The relationship is 1143 categorised by an enumerated value from a fixed vocabulary. 1145 The relationship_type is recommended to be taken from the MISP object 1146 relationship list [[MISP-R]] is RECOMMENDED to ensure a coherent 1147 naming of the tags 1149 All Object References MUST contain an object_uuid, a referenced_uuid 1150 and a relationship type. 1152 2.7.1. Sample ObjectReference object 1153 "ObjectReference": { 1154 "id": "195", 1155 "uuid": "59c21a2c-c0ac-4083-93b3-363da07724d1", 1156 "timestamp": "1505892908", 1157 "object_id": "591", 1158 "event_id": "113", 1159 "referenced_id": "590", 1160 "referenced_type": "1", 1161 "relationship_type": "derived-from", 1162 "comment": "", 1163 "deleted": false, 1164 "object_uuid": "59c1134d-8a40-4c14-ad94-0f7ba07724d1", 1165 "referenced_uuid": "59c1133c-9adc-4d06-a34b-0f7ca07724d1", 1166 } 1168 2.7.2. ObjectReference Attributes 1170 2.7.2.1. uuid 1172 uuid represents the Universally Unique IDentifier (UUID) [RFC4122] of 1173 the object reference. The uuid MUST be preserved for any updates or 1174 transfer of the same object reference. UUID version 4 is RECOMMENDED 1175 when assigning it to a new object reference. 1177 2.7.2.2. id 1179 id represents the human-readable identifier associated to the object 1180 reference for a specific MISP instance. 1182 id is represented as a JSON string. id SHALL be present. 1184 2.7.2.3. timestamp 1186 timestamp represents a reference time when the object was created or 1187 last modified. timestamp is expressed in seconds (decimal) since 1st 1188 of January 1970 (Unix timestamp). The time zone MUST be UTC. 1190 timestamp is represented as a JSON string. timestamp MUST be present. 1192 2.7.2.4. object_id 1194 object_id represents the human-readable identifier of the object that 1195 the object reference belongs to on a specific MISP instance. A 1196 human-readable identifier MUST be represented as an unsigned integer. 1198 event_id is represented as a JSON string. event_id SHALL be present. 1200 2.7.2.5. event_id 1202 event_id represents the human-readable identifier of the event that 1203 the object reference belongs to on a specific MISP instance. A 1204 human-readable identifier MUST be represented as an unsigned integer. 1206 event_id is represented as a JSON string. event_id SHALL be present. 1208 2.7.2.6. referenced_id 1210 referenced_id represents the human-readable identifier of the object 1211 or attribute that the parent object of the object reference points to 1212 on a specific MISP instance. 1214 referenced_id is represented as a JSON string. referenced_id MAY be 1215 present. 1217 2.7.2.7. referenced_type 1219 referenced_type represents the numeric value describing what the 1220 object reference points to, "0" representing an attribute and "1" 1221 representing an object 1223 referenced_type is represented as a JSON string. referenced_type MAY 1224 be present. 1226 2.7.2.8. relationship_type 1228 relationship_type represents the human-readable context of the 1229 relationship between an object and another object or attribute as 1230 described by the object_reference. 1232 referenced_type is represented as a JSON string. relationship_type 1233 MUST be present. 1235 2.7.2.9. comment 1237 comment is a contextual comment field. 1239 comment is represented by a JSON string. comment MAY be present. 1241 2.7.2.10. deleted 1243 deleted represents a setting that allows object references to be 1244 revoked. Revoked object references are not actionable and exist 1245 merely to inform other instances of a revocation. 1247 deleted is represented by a JSON boolean. deleted MUST be present. 1249 2.7.2.11. object_uuid 1251 object_uuid represents the Universally Unique IDentifier (UUID) 1252 [RFC4122] of the object that the given object reference belongs to. 1253 The object_uuid MUST be preserved to preserve the object reference's 1254 association with the object. 1256 2.7.2.12. referenced_uuid 1258 referenced_uuid represents the Universally Unique IDentifier (UUID) 1259 [RFC4122] of the object or attribute that is being referenced by the 1260 object reference. The referenced_uuid MUST be preserved to preserve 1261 the object reference's association with the object or attribute. 1263 2.8. Tag 1265 A tag is a simple method to classify an event with a simple string. 1266 The tag name can be freely chosen. The tag name can be also chosen 1267 from a fixed machine-tag vocabulary called MISP taxonomies[[MISP-T]]. 1268 When an event is distributed outside an organisation, the use of MISP 1269 taxonomies[[MISP-T]] is RECOMMENDED to ensure a coherent naming of 1270 the tags. A tag is represented as a JSON array where each element 1271 describes each tag associated. A tag array SHALL be at event level 1272 or attribute level. A tag element is described with a name, id, 1273 colour and exportable flag. 1275 exportable represents a setting if the tag is kept local or 1276 exportable to other MISP instances. exportable is represented by a 1277 JSON boolean. id is a human-readable identifier that references the 1278 tag on the local instance. colour represents an RGB value of the tag. 1280 name MUST be present. colour, id and exportable SHALL be present. 1282 2.8.1. Sample Tag 1284 "Tag": [{ 1285 "exportable": true, 1286 "colour": "#ffffff", 1287 "name": "tlp:white", 1288 "id": "2" }] 1290 2.9. Sighting 1292 A sighting is an ascertainment which describes whether an attribute 1293 has been seen under a given set of conditions. The sighting can 1294 include the organisation who sighted the attribute or can be 1295 anonymised. Sighting is composed of a JSON array in which each 1296 element describes one singular instance of a sighting. A sighting 1297 element is a JSON object composed of the following values: 1299 type MUST be present. type describes the type of a sighting. MISP 1300 allows 3 default types: 1302 +------------+------------------------------------------------------+ 1303 | Sighting | Description | 1304 | type | | 1305 +------------+------------------------------------------------------+ 1306 | 0 | denotes an attribute which has been seen | 1307 | 1 | denotes an attribute which has been seen and | 1308 | | confirmed as false-positive | 1309 | 2 | denotes an attribute which will be expired at the | 1310 | | time of the sighting | 1311 +------------+------------------------------------------------------+ 1313 uuid MUST be present. uuid references the uuid of the sighted 1314 attribute. 1316 date_sighting MUST be present. date_sighting is expressed in seconds 1317 (decimal) elapsed since 1st of January 1970 (Unix timestamp). 1318 date_sighting represents when the referenced attribute, designated by 1319 its uuid, is sighted. 1321 source MAY be present. source is represented as a JSON string and 1322 represents the human-readable version of the sighting source, which 1323 can be a given piece of software (e.g. SIEM), device or a specific 1324 analytical process. 1326 id, event_id and attribute_id MAY be present. 1328 id represents the human-readable identifier of the sighting reference 1329 which belongs to a specific MISP instance. event_id represents the 1330 human-readable identifier of the event referenced by the sighting and 1331 belongs to a specific MISP instance. attribute_id represents the 1332 human-readable identifier of the attribute referenced by the sighting 1333 and belongs to a specific MISP instance. 1335 org_id MAY be present along the JSON object describing the 1336 organisation. If the org_id is not present, the sighting is 1337 considered as anonymised. 1339 org_id represents the human-readable identifier of the organisation 1340 which did the sighting and belongs to a specific MISP instance. 1342 A human-readable identifier MUST be represented as an unsigned 1343 integer. 1345 2.9.1. Sample Sighting 1347 "Sighting": [ 1348 { 1349 "id": "13599", 1350 "attribute_id": "1201615", 1351 "event_id": "10164", 1352 "org_id": "2", 1353 "date_sighting": "1517581400", 1354 "uuid": "5a747459-41b4-4826-9b29-42dd950d210f", 1355 "source": "M2M-CIRCL", 1356 "type": "0", 1357 "Organisation": { 1358 "id": "2", 1359 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f", 1360 "name": "CIRCL" 1361 } 1362 }, 1363 { 1364 "id": "13601", 1365 "attribute_id": "1201615", 1366 "event_id": "10164", 1367 "org_id": "2", 1368 "date_sighting": "1517581401", 1369 "uuid": "5a74745a-a190-4d04-b719-4916950d210f", 1370 "source": "M2M-CIRCL", 1371 "type": "0", 1372 "Organisation": { 1373 "id": "2", 1374 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f", 1375 "name": "CIRCL" 1376 } 1377 } 1378 ] 1380 2.10. Galaxy 1382 A galaxy is a simple method to express a large object called cluster 1383 that can be attached to MISP events. A cluster can be composed of 1384 one or more elements. Elements are expressed as key-values. 1386 2.10.1. Sample Galaxy 1387 "Galaxy": [ { 1388 "id": "18", 1389 "uuid": "698774c7-8022-42c4-917f-8d6e4f06ada3", 1390 "name": "Threat Actor", 1391 "type": "threat-actor", 1392 "description": "Threat actors are characteristics of malicious actors 1393 (or adversaries) representing a cyber attack threat 1394 including presumed intent and historically observed behaviour.", 1395 "version": "1", 1396 "GalaxyCluster": [ 1397 { 1398 "id": "1699", 1399 "uuid": "7cdff317-a673-4474-84ec-4f1754947823", 1400 "type": "threat-actor", 1401 "value": "Anunak", 1402 "tag_name": "misp-galaxy:threat-actor=\"Anunak\"", 1403 "description": "Groups targeting financial organizations 1404 or people with significant financial assets.", 1405 "galaxy_id": "18", 1406 "source": "MISP Project", 1407 "authors": [ 1408 "Alexandre Dulaunoy", 1409 "Florian Roth", 1410 "Thomas Schreck", 1411 "Timo Steffens", 1412 "Various" 1413 ], 1414 "tag_id": "111", 1415 "meta": { 1416 "synonyms": [ 1417 "Carbanak", 1418 "Carbon Spider" 1419 ], 1420 "country": [ 1421 "RU" 1422 ], 1423 "motive": [ 1424 "Cybercrime" 1425 ] 1426 } 1427 } 1428 ] 1429 } 1430 ] 1432 3. JSON Schema 1434 The JSON Schema [JSON-SCHEMA] below defines the structure of the MISP 1435 core format as literally described before. The JSON Schema is used 1436 to validate MISP events at creation time or parsing. 1438 { 1439 "$schema": "http://json-schema.org/draft-04/schema#", 1440 "title": "Validator for misp events", 1441 "id": "https://github.com/MISP/MISP/blob/2.4/format/2.4/schema.json", 1442 "defs": { 1443 "org": { 1444 "type": "object", 1445 "additionalProperties": false, 1446 "properties": { 1447 "id": { 1448 "type": "string" 1449 }, 1450 "name": { 1451 "type": "string" 1452 }, 1453 "uuid": { 1454 "type": "string" 1455 } 1456 }, 1457 "required": [ 1458 "uuid" 1459 ] 1460 }, 1461 "orgc": { 1462 "type": "object", 1463 "additionalProperties": false, 1464 "properties": { 1465 "id": { 1466 "type": "string" 1467 }, 1468 "name": { 1469 "type": "string" 1470 }, 1471 "uuid": { 1472 "type": "string" 1473 } 1474 }, 1475 "required": [ 1476 "uuid" 1477 ] 1478 }, 1479 "sharing_group": { 1480 "type": "object", 1481 "additionalProperties": false, 1482 "properties": { 1483 "id": { 1484 "type": "string" 1485 }, 1486 "name": { 1487 "type": "string" 1488 }, 1489 "releasability": { 1490 "type": "string" 1491 }, 1492 "description": { 1493 "type": "string" 1494 }, 1495 "uuid": { 1496 "type": "string" 1497 }, 1498 "organisation_uuid": { 1499 "type": "string" 1500 }, 1501 "org_id": { 1502 "type": "string" 1503 }, 1504 "sync_user_id": { 1505 "type": "string" 1506 }, 1507 "active": { 1508 "type": "boolean" 1509 }, 1510 "created": { 1511 "type": "string" 1512 }, 1513 "modified": { 1514 "type": "string" 1515 }, 1516 "local": { 1517 "type": "boolean" 1518 }, 1519 "roaming": { 1520 "type": "boolean" 1521 }, 1522 "Organisation": { 1523 "$ref": "#/defs/org" 1524 }, 1525 "SharingGroupOrg": { 1526 "type": "array", 1527 "uniqueItems": true, 1528 "items": { 1529 "$ref": "#/defs/sharing_group_org" 1530 } 1531 }, 1532 "SharingGroupServer": { 1533 "type": "array", 1534 "uniqueItems": true, 1535 "items": { 1536 "$ref": "#/defs/sharing_group_server" 1537 } 1538 }, 1539 "required": [ 1540 "uuid" 1541 ] 1542 }, 1543 "required": [ 1544 "uuid" 1545 ] 1546 }, 1547 "sharing_group_org": { 1548 "type": "object", 1549 "additionalProperties": false, 1550 "properties": { 1551 "id": { 1552 "type": "string" 1553 }, 1554 "sharing_group_id": { 1555 "type": "string" 1556 }, 1557 "org_id": { 1558 "type": "string" 1559 }, 1560 "extend": { 1561 "type": "boolean" 1562 }, 1563 "Organisation": { 1564 "$ref": "#/defs/org" 1565 } 1566 } 1567 }, 1568 "sharing_group_server": { 1569 "type": "object", 1570 "additionalProperties": false, 1571 "properties": { 1572 "id": { 1573 "type": "string" 1574 }, 1575 "sharing_group_id": { 1576 "type": "string" 1577 }, 1578 "server_id": { 1579 "type": "string" 1580 }, 1581 "all_orgs": { 1582 "type": "boolean" 1583 }, 1584 "Server": { 1585 "$ref": "#/defs/server" 1586 } 1587 } 1588 }, 1589 "server": { 1590 "type": "object", 1591 "additionalProperties": false, 1592 "properties": { 1593 "id": { 1594 "type": "string" 1595 }, 1596 "url": { 1597 "type": "string" 1598 }, 1599 "name": { 1600 "type": "string" 1601 } 1602 } 1603 }, 1604 "attribute": { 1605 "type": "object", 1606 "additionalProperties": false, 1607 "properties": { 1608 "id": { 1609 "type": "string" 1610 }, 1611 "type": { 1612 "type": "string" 1613 }, 1614 "category": { 1615 "type": "string" 1616 }, 1617 "to_ids": { 1618 "type": "boolean" 1619 }, 1620 "uuid": { 1621 "type": "string" 1622 }, 1623 "event_id": { 1624 "type": "string" 1625 }, 1626 "distribution": { 1627 "type": "string" 1628 }, 1629 "timestamp": { 1630 "type": "string" 1631 }, 1632 "comment": { 1633 "type": "string" 1634 }, 1635 "sharing_group_id": { 1636 "type": "string" 1637 }, 1638 "deleted": { 1639 "type": "boolean" 1640 }, 1641 "disable_correlation": { 1642 "type": "boolean" 1643 }, 1644 "value": { 1645 "type": "string" 1646 }, 1647 "data": { 1648 "type": "string" 1649 }, 1650 "SharingGroup": { 1651 "$ref": "#/defs/sharing_group" 1652 }, 1653 "ShadowAttribute": { 1654 "type": "array", 1655 "uniqueItems": true, 1656 "items": { 1657 "$ref": "#/defs/attribute" 1658 } 1659 }, 1660 "Tag": { 1661 "type": "array", 1662 "uniqueItems": true, 1663 "items": { 1664 "$ref": "#/defs/tag" 1665 } 1666 } 1667 } 1668 }, 1669 "event": { 1670 "type": "object", 1671 "additionalProperties": false, 1672 "properties": { 1673 "id": { 1674 "type": "string" 1675 }, 1676 "orgc_id": { 1677 "type": "string" 1678 }, 1679 "org_id": { 1680 "type": "string" 1681 }, 1682 "date": { 1683 "type": "string" 1684 }, 1685 "threat_level_id": { 1686 "type": "string" 1687 }, 1688 "info": { 1689 "type": "string" 1690 }, 1691 "published": { 1692 "type": "boolean" 1693 }, 1694 "uuid": { 1695 "type": "string" 1696 }, 1697 "attribute_count": { 1698 "type": "string" 1699 }, 1700 "analysis": { 1701 "type": "string" 1702 }, 1703 "timestamp": { 1704 "type": "string" 1705 }, 1706 "distribution": { 1707 "type": "string" 1708 }, 1709 "proposal_email_lock": { 1710 "type": "boolean" 1711 }, 1712 "locked": { 1713 "type": "boolean" 1714 }, 1715 "publish_timestamp": { 1716 "type": "string" 1717 }, 1718 "sharing_group_id": { 1719 "type": "string" 1721 }, 1722 "disable_correlation": { 1723 "type": "boolean" 1724 }, 1725 "event_creator_email": { 1726 "type": "string" 1727 }, 1728 "Org": { 1729 "$ref": "#/defs/org" 1730 }, 1731 "Orgc": { 1732 "$ref": "#/defs/org" 1733 }, 1734 "SharingGroup": { 1735 "$ref": "#/defs/sharing_group" 1736 }, 1737 "Attribute": { 1738 "type": "array", 1739 "uniqueItems": true, 1740 "items": { 1741 "$ref": "#/defs/attribute" 1742 } 1743 }, 1744 "ShadowAttribute": { 1745 "type": "array", 1746 "uniqueItems": true, 1747 "items": { 1748 "$ref": "#/defs/attribute" 1749 } 1750 }, 1751 "RelatedEvent": { 1752 "type": "array", 1753 "uniqueItems": true, 1754 "items": { 1755 "type": "object", 1756 "additionalProperties": false, 1757 "properties": { 1758 "Event":{ 1759 "$ref": "#/defs/event" 1760 } 1761 } 1762 } 1763 }, 1764 "Galaxy": { 1765 "type": "array", 1766 "uniqueItems": true, 1767 "items": { 1768 "$ref": "#/defs/galaxy" 1770 } 1771 }, 1772 "Tag": { 1773 "type": "array", 1774 "uniqueItems": true, 1775 "items": { 1776 "$ref": "#/defs/tag" 1777 } 1778 } 1779 } 1780 }, 1781 "tag": { 1782 "type": "object", 1783 "additionalProperties": false, 1784 "properties": { 1785 "id": { 1786 "type": "string" 1787 }, 1788 "name": { 1789 "type": "string" 1790 }, 1791 "colour": { 1792 "type": "string" 1793 }, 1794 "exportable": { 1795 "type": "boolean" 1796 }, 1797 "hide_tag": { 1798 "type": "boolean" 1799 } 1800 } 1801 }, 1802 "galaxy": { 1803 "type": "object", 1804 "additionalProperties": false, 1805 "properties": { 1806 "id": { 1807 "type": "string" 1808 }, 1809 "uuid": { 1810 "type": "string" 1811 }, 1812 "name": { 1813 "type": "string" 1814 }, 1815 "type": { 1816 "type": "string" 1817 }, 1818 "description": { 1819 "type": "string" 1820 }, 1821 "version": { 1822 "type": "string" 1823 }, 1824 "GalaxyCluster": { 1825 "type": "array", 1826 "uniqueItems": true, 1827 "items": { 1828 "$ref": "#/defs/galaxy_cluster" 1829 } 1830 } 1831 } 1832 }, 1833 "galaxy_cluster": { 1834 "type": "object", 1835 "additionalProperties": false, 1836 "properties": { 1837 "id": { 1838 "type": "string" 1839 }, 1840 "uuid": { 1841 "type": "string" 1842 }, 1843 "type": { 1844 "type": "string" 1845 }, 1846 "value": { 1847 "type": "string" 1848 }, 1849 "tag_name": { 1850 "type": "string" 1851 }, 1852 "description": { 1853 "type": "string" 1854 }, 1855 "galaxy_id": { 1856 "type": "string" 1857 }, 1858 "source": { 1859 "type": "string" 1860 }, 1861 "authors": { 1862 "type": "array", 1863 "uniqueItems": true, 1864 "items": { 1865 "type": "string" 1867 } 1868 }, 1869 "tag_id": { 1870 "type": "string" 1871 }, 1872 "meta": { 1873 "type": "object" 1874 } 1875 } 1876 } 1877 }, 1878 "type": "object", 1879 "properties": { 1880 "Event": { 1881 "$ref": "#/defs/event" 1882 } 1883 }, 1884 "required": [ 1885 "Event" 1886 ] 1887 } 1889 4. Manifest 1891 MISP events can be shared over an HTTP repository, a file package or 1892 USB key. A manifest file is used to provide an index of MISP events 1893 allowing to only fetch the recently updated files without the need to 1894 parse each json file. 1896 4.1. Format 1898 A manifest file is a simple JSON file named manifest.json in a 1899 directory where the MISP events are located. Each MISP event is a 1900 file located in the same directory with the event uuid as filename 1901 with the json extension. 1903 The manifest format is a JSON object composed of a dictionary where 1904 the field is the uuid of the event. 1906 Each uuid is composed of a JSON object with the following fields 1907 which came from the original event referenced by the same uuid: 1909 o info (MUST) 1911 o Orgc object (MUST) 1913 o analysis (SHALL) 1914 o timestamp (MUST) 1916 o date (MUST) 1918 o threat_level_id (SHALL) 1920 In addition to the fields originating from the event, the following 1921 fields can be added: 1923 o integrity:sha256 represents the SHA256 value in hexadecimal 1924 representation of the associated MISP event file to ensure 1925 integrity of the file. (SHOULD) 1927 o integrity:pgp represents a detached PGP signature [RFC4880] of the 1928 associated MISP event file to ensure integrity of the file. 1929 (SHOULD) 1931 If a detached PGP signature is used for each MISP event, a detached 1932 PGP signature is a MUST to ensure integrity of the manifest file. A 1933 detached PGP signature for a manifest file is a manifest.json.asc 1934 file containing the PGP signature. 1936 4.1.1. Sample Manifest 1938 { 1939 "57c6ac4c-c60c-4f79-a38f-b666950d210f": { 1940 "info": "Malspam 2016-08-31 (.wsf in .zip) - campaign: Photo", 1941 "Orgc": { 1942 "id": "2", 1943 "name": "CIRCL", 1944 "uuid": "55f6ea5e-2c60-40e5-964f-47a8950d210f" 1945 }, 1946 "analysis": "0", 1947 "Tag": [ 1948 { 1949 "colour": "#3d7a00", 1950 "name": "circl:incident-classification=\"malware\"" 1951 }, 1952 { 1953 "colour": "#ffffff", 1954 "name": "tlp:white" 1955 } 1956 ], 1957 "timestamp": "1472638251", 1958 "date": "2016-08-31", 1959 "threat_level_id": "3" 1960 }, 1961 "5720accd-dd28-45f8-80e5-4605950d210f": { 1962 "info": "Malspam 2016-04-27 - Locky", 1963 "Orgc": { 1964 "id": "2", 1965 "name": "CIRCL" 1966 }, 1967 "analysis": "2", 1968 "Tag": [ 1969 { 1970 "colour": "#ffffff", 1971 "name": "tlp:white" 1972 }, 1973 { 1974 "colour": "#3d7a00", 1975 "name": "circl:incident-classification=\"malware\"" 1976 }, 1977 { 1978 "colour": "#2c4f00", 1979 "name": "malware_classification:malware-category=\"Ransomware\"" 1980 } 1981 ], 1982 "timestamp": "1461764231", 1983 "date": "2016-04-27", 1984 "threat_level_id": "3" 1985 } 1986 } 1988 5. Implementation 1990 MISP format is implemented by different software including the MISP 1991 threat sharing platform and libraries like PyMISP [MISP-P]. 1992 Implementations use the format as an export/import mechanism, staging 1993 transport format or synchronisation format as used in the MISP core 1994 platform. MISP format doesn't impose any restriction on the data 1995 representation of the format in data-structure of other 1996 implementations. 1998 6. Security Considerations 2000 MISP events might contain sensitive or confidential information. 2001 Adequate access control and encryption measures shall be implemented 2002 to ensure the confidentiality of the MISP events. 2004 Adversaries might include malicious content in MISP events and 2005 attributes. Implementation MUST consider the input of malicious 2006 inputs beside the standard threat information that might already 2007 include malicious intended inputs. 2009 7. Acknowledgements 2011 The authors wish to thank all the MISP community who are supporting 2012 the creation of open standards in threat intelligence sharing. 2014 8. Sample MISP file 2016 9. References 2018 9.1. Normative References 2020 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2021 Requirement Levels", BCP 14, RFC 2119, 2022 DOI 10.17487/RFC2119, March 1997, 2023 . 2025 [RFC4122] Leach, P., Mealling, M., and R. Salz, "A Universally 2026 Unique IDentifier (UUID) URN Namespace", RFC 4122, 2027 DOI 10.17487/RFC4122, July 2005, 2028 . 2030 [RFC4627] Crockford, D., "The application/json Media Type for 2031 JavaScript Object Notation (JSON)", RFC 4627, 2032 DOI 10.17487/RFC4627, July 2006, 2033 . 2035 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 2036 Thayer, "OpenPGP Message Format", RFC 4880, 2037 DOI 10.17487/RFC4880, November 2007, 2038 . 2040 9.2. Informative References 2042 [JSON-SCHEMA] 2043 "JSON Schema: A Media Type for Describing JSON Documents", 2044 2016, 2045 . 2047 [MISP-P] MISP, "MISP Project - Malware Information Sharing Platform 2048 and Threat Sharing", . 2050 [MISP-R] MISP, "MISP Object Relationship Types - common vocabulary 2051 of relationships", . 2054 [MISP-T] MISP, "MISP Taxonomies - shared and common vocabularies of 2055 tags", . 2057 Authors' Addresses 2059 Alexandre Dulaunoy 2060 Computer Incident Response Center Luxembourg 2061 16, bd d'Avranches 2062 Luxembourg L-1160 2063 Luxembourg 2065 Phone: +352 247 88444 2066 Email: alexandre.dulaunoy@circl.lu 2068 Andras Iklody 2069 Computer Incident Response Center Luxembourg 2070 16, bd d'Avranches 2071 Luxembourg L-1160 2072 Luxembourg 2074 Phone: +352 247 88444 2075 Email: andras.iklody@circl.lu