idnits 2.17.1 draft-hallambaker-mesh-dare-06.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Authors' Addresses Section. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (16 January 2020) is 1533 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 7159 (Obsoleted by RFC 8259) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. M. Hallam-Baker 3 Internet-Draft 16 January 2020 4 Intended status: Informational 5 Expires: 19 July 2020 7 Mathematical Mesh 3.0 Part III : Data At Rest Encryption (DARE) 8 draft-hallambaker-mesh-dare-06 10 Abstract 12 This document describes the Data At Rest Encryption (DARE) Envelope 13 and Container syntax. 15 The DARE Envelope syntax is used to digitally sign, digest, 16 authenticate, or encrypt arbitrary content data. 18 The DARE Container syntax describes an append-only sequence of 19 entries, each containing a DARE Envelope. DARE Containers may 20 support cryptographic integrity verification of the entire data 21 container content by means of a Merkle tree. 23 [Note to Readers] 25 Discussion of this draft takes place on the MATHMESH mailing list 26 (mathmesh@ietf.org), which is archived at 27 https://mailarchive.ietf.org/arch/search/?email_list=mathmesh. 29 This document is also available online at 30 http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at https://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on 19 July 2020. 49 Copyright Notice 51 Copyright (c) 2020 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 56 license-info) in effect on the date of publication of this document. 57 Please review these documents carefully, as they describe your rights 58 and restrictions with respect to this document. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 63 1.1. Encryption and Integrity . . . . . . . . . . . . . . . . 5 64 1.1.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 5 65 1.1.2. Data Erasure . . . . . . . . . . . . . . . . . . . . 6 66 1.2. Signature . . . . . . . . . . . . . . . . . . . . . . . . 6 67 1.2.1. Signing Individual Plaintext Envelopes . . . . . . . 7 68 1.2.2. Signing Individual Encrypted Envelopes . . . . . . . 7 69 1.2.3. Signing sequences of envelopes . . . . . . . . . . . 7 70 1.3. Container . . . . . . . . . . . . . . . . . . . . . . . . 8 71 1.3.1. Container Format . . . . . . . . . . . . . . . . . . 8 72 1.3.2. Write . . . . . . . . . . . . . . . . . . . . . . . . 9 73 1.3.3. Encryption and Authentication . . . . . . . . . . . . 9 74 1.3.4. Integrity and Signature . . . . . . . . . . . . . . . 10 75 1.3.5. Redaction . . . . . . . . . . . . . . . . . . . . . . 10 76 1.3.6. Alternative approaches . . . . . . . . . . . . . . . 11 77 1.3.7. Efficiency . . . . . . . . . . . . . . . . . . . . . 11 78 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 12 79 2.1. Related Specifications . . . . . . . . . . . . . . . . . 12 80 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 12 81 2.3. Defined terms . . . . . . . . . . . . . . . . . . . . . . 13 82 3. DARE Envelope Architecture . . . . . . . . . . . . . . . . . 14 83 3.1. Processing Considerations . . . . . . . . . . . . . . . . 15 84 3.2. Content Metadata and Annotations . . . . . . . . . . . . 15 85 3.3. Encoded Data Sequence . . . . . . . . . . . . . . . . . . 16 86 3.4. Encryption and Integrity . . . . . . . . . . . . . . . . 17 87 3.4.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 17 88 3.4.2. Key Identifiers . . . . . . . . . . . . . . . . . . . 18 89 3.4.3. Salt Derivation . . . . . . . . . . . . . . . . . . . 19 90 3.4.4. Key Derivation . . . . . . . . . . . . . . . . . . . 19 91 3.5. Signature . . . . . . . . . . . . . . . . . . . . . . . . 20 92 3.6. Algorithms . . . . . . . . . . . . . . . . . . . . . . . 20 93 3.6.1. Field: kwd . . . . . . . . . . . . . . . . . . . . . 20 94 4. DARE Container Architecture . . . . . . . . . . . . . . . . . 20 95 4.1. Container Navigation . . . . . . . . . . . . . . . . . . 21 96 4.1.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . 22 97 4.1.2. Position Index . . . . . . . . . . . . . . . . . . . 22 98 4.1.3. Metadata Index . . . . . . . . . . . . . . . . . . . 22 99 4.2. Integrity Mechanisms . . . . . . . . . . . . . . . . . . 22 100 4.2.1. Digest Chain calculation . . . . . . . . . . . . . . 23 101 4.2.2. Binary Merkle tree calculation . . . . . . . . . . . 23 102 4.2.3. Signature . . . . . . . . . . . . . . . . . . . . . . 23 103 5. DARE Schema . . . . . . . . . . . . . . . . . . . . . . . . . 24 104 5.1. Message Classes . . . . . . . . . . . . . . . . . . . . . 24 105 5.1.1. Structure: DareEnvelopeSequence . . . . . . . . . . . 24 106 5.2. Header and Trailer Classes . . . . . . . . . . . . . . . 25 107 5.2.1. Structure: DareTrailer . . . . . . . . . . . . . . . 25 108 5.2.2. Structure: DareHeader . . . . . . . . . . . . . . . . 25 109 5.2.3. Structure: ContentMeta . . . . . . . . . . . . . . . 26 110 5.3. Cryptographic Data . . . . . . . . . . . . . . . . . . . 27 111 5.3.1. Structure: DareSigner . . . . . . . . . . . . . . . . 27 112 5.3.2. Structure: X509Certificate . . . . . . . . . . . . . 28 113 5.3.3. Structure: DareSignature . . . . . . . . . . . . . . 28 114 5.3.4. Structure: DareRecipient . . . . . . . . . . . . . . 28 115 6. DARE Container Schema . . . . . . . . . . . . . . . . . . . . 28 116 6.1. Container Headers . . . . . . . . . . . . . . . . . . . . 29 117 6.1.1. Structure: ContainerInfo . . . . . . . . . . . . . . 29 118 6.2. Index Structures . . . . . . . . . . . . . . . . . . . . 29 119 6.2.1. Structure: ContainerIndex . . . . . . . . . . . . . . 29 120 6.2.2. Structure: IndexPosition . . . . . . . . . . . . . . 30 121 6.2.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 30 122 6.2.4. Structure: IndexMeta . . . . . . . . . . . . . . . . 30 123 7. Dare Container Applications . . . . . . . . . . . . . . . . . 31 124 7.1. Catalog . . . . . . . . . . . . . . . . . . . . . . . . . 31 125 7.2. Spool . . . . . . . . . . . . . . . . . . . . . . . . . . 31 126 7.3. Archive . . . . . . . . . . . . . . . . . . . . . . . . . 32 127 8. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 32 128 8.1. Terminal integrity check . . . . . . . . . . . . . . . . 32 129 8.2. Terminal index record . . . . . . . . . . . . . . . . . . 33 130 8.3. Deferred indexing . . . . . . . . . . . . . . . . . . . . 33 131 9. Security Considerations . . . . . . . . . . . . . . . . . . . 33 132 9.1. Encryption/Signature nesting . . . . . . . . . . . . . . 34 133 9.2. Side channel . . . . . . . . . . . . . . . . . . . . . . 34 134 9.3. Salt reuse . . . . . . . . . . . . . . . . . . . . . . . 34 135 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 136 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34 137 12. Appendix A: DARE Envelope Examples and Test Vectors . . . . . 34 138 13. Test Examples . . . . . . . . . . . . . . . . . . . . . . . . 34 139 13.1. Plaintext Message . . . . . . . . . . . . . . . . . . . 35 140 13.2. Plaintext Message with EDS . . . . . . . . . . . . . . . 35 141 13.3. Encrypted Message . . . . . . . . . . . . . . . . . . . 35 142 13.4. Signed Message . . . . . . . . . . . . . . . . . . . . . 37 143 13.5. Signed and Encrypted Message . . . . . . . . . . . . . . 37 144 14. Appendix B: DARE Container Examples and Test Vectors . . . . 38 145 14.1. Simple container . . . . . . . . . . . . . . . . . . . . 38 146 14.2. Payload and chain digests . . . . . . . . . . . . . . . 39 147 14.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . 41 148 14.4. Signed container . . . . . . . . . . . . . . . . . . . . 43 149 14.5. Encrypted container . . . . . . . . . . . . . . . . . . 44 150 15. Appendix C: Previous Frame Function . . . . . . . . . . . . . 47 151 16. Appendix D: Outstanding Issues . . . . . . . . . . . . . . . 47 152 17. Normative References . . . . . . . . . . . . . . . . . . . . 48 153 18. Informative References . . . . . . . . . . . . . . . . . . . 49 155 1. Introduction 157 This document describes the Data At Rest Encryption (DARE) Envelope 158 and Container Syntax. The DARE Envelope syntax is used to digitally 159 sign, digest, authenticate, or encrypt arbitrary message content. 160 The DARE Container syntax describes an append-only sequence of data 161 frames, each containing a DARE Envelope that supports efficient 162 incremental signature and encryption. 164 The DARE Envelope Syntax is based on a subset of the JSON Web 165 Signature [RFC7515] and JSON Web Encryption [RFC7516] standards and 166 shares many fields and semantics. The processing model and data 167 structures have been streamlined to remove alternative means of 168 specifying the same content and to enable multiple data sequences to 169 be signed and encrypted under a single master encryption key without 170 compromise to security. 172 (Artwork only available as svg: No external link available, see 173 draft-hallambaker-mesh-dare-06.html for artwork.) 175 Figure 1 177 A DARE Envelope consists of a _Header_, _Payload_ and an optional 178 _Trailer_. To enable single pass encoding and decoding, the Header 179 contains all the information required to perform cryptographic 180 processing of the Payload and authentication data (digest, MAC, 181 signature values) MAY be deferred to the Trailer section. 183 A DARE Container is an append-only log format consisting of a 184 sequence of frames. Cryptographic enhancements (signature, 185 encryption) may be applied to individual frames or to sets of frames. 186 Thus, a single key exchange may be used to provide a master key to 187 encrypt multiple frames and a single signature may be used to 188 authenticate all the frames in the container up to and including the 189 frame in which the signature is presented. 191 The DARE Envelope syntax may be used either as a standalone 192 cryptographic message syntax or as a means of presenting a single 193 DARE Container frame together with the complete cryptographic context 194 required to verify the contents and decrypt them. 196 1.1. Encryption and Integrity 198 A key innovation in the DARE Envelope Syntax is the separation of key 199 exchange and data encryption operations so that a Master Key (MK) 200 established in a single exchange to be applied to multiple data 201 sequences. This means that a single public key operation MAY be used 202 to encrypt and/or authenticate multiple parts of the same DARE 203 Envelope or multiple frames in a DARE Container. 205 To avoid reuse of the key and to avoid the need to communicate 206 separate IVs, each octet sequence is encrypted under a different 207 encryption key (and IV if required) derived from the Master Key by 208 means of a salt that is unique for each octet sequence that is 209 encrypted. The same approach is used to generate keys for 210 calculating a MAC over the octet sequence if required. This approach 211 allows encryption and integrity protections to be applied to the 212 envelope payload, to header or trailer fields or to application 213 defined Enhanced Data Sequences in the header or trailer. 215 1.1.1. Key Exchange 217 Traditional cryptographic containers describe the application of a 218 single key exchange to encryption of a single octet sequence. 219 Examples include PCKS#7/CMS [RFC2315], OpenPGP [RFC4880] and JSON Web 220 Encryption [RFC7516]. 222 To encrypt data using RSA, the encoder first generates a random 223 encryption key and initialization vector (IV). The encryption key is 224 encrypted under the public key of each recipient to create a per- 225 recipient decryption entry. The encryption key, plaintext and IV are 226 used to generate the ciphertext (figure 1). 228 (Artwork only available as svg: No external link available, see 229 draft-hallambaker-mesh-dare-06.html for artwork.) 231 Figure 2 233 This approach is adequate for the task of encrypting a single octet 234 stream. It is less than satisfactory when encrypting multiple octet 235 streams or very long streams for which a rekeying operation is 236 desirable. 238 In the DARE approach, key exchange and key derivation are separate 239 operations and keys MAY be derived for encryption or integrity 240 purposes or both. A single key exchange MAY be used to derive keys 241 to apply encryption and integrity enhancements to multiple data 242 sequences. 244 The DARE key exchange begins with the same key exchange used to 245 produce the CEK in JWE but instead of using the CEK to encipher data 246 directly, it is used as one of the inputs to a Key Derivation 247 Function (KDF) that is used to derive parameters for each block of 248 data to be encrypted. To avoid the need to introduce additional 249 terminology, the term 'CEK' is still used to describe the output of 250 the key agreement algorithm (including key unwrapping if required) 251 but it is more appropriately described as a Master Key (figure 2). 253 (Artwork only available as svg: No external link available, see 254 draft-hallambaker-mesh-dare-06.html for artwork.) 256 Figure 3 258 A Master Key may be used to encrypt any number of data items. Each 259 data item is encrypted under a different encryption key and IV (if 260 required). This data is derived from the Master Key using the HKDF 261 function [RFC5869] using a different salt for each data item and 262 separate info tags for each cryptographic function (figure 3). 264 (Artwork only available as svg: No external link available, see 265 draft-hallambaker-mesh-dare-06.html for artwork.) 267 Figure 4 269 This approach to encryption offers considerably greater flexibility 270 allowing the same format for data item encryption to be applied at 271 the transport, message or field level. 273 1.1.2. Data Erasure 275 Each encrypted DARE Envelope specifies a unique Master Salt value of 276 at least 128 bits which is used to derive the salt values used to 277 derive cryptographic keys for the envelope payload and annotations. 279 Erasure of the Master Salt value MAY be used to effectively render 280 the envelope payload and annotations undecipherable without altering 281 the envelope payload data. The work factor for decryption will be 282 O(2^128) even if the decryption key is compromised. 284 1.2. Signature 286 As with encryption, DARE Envelope signatures MAY be applied to an 287 individual envelope or a sequence of envelope. 289 1.2.1. Signing Individual Plaintext Envelopes 291 When an individual plaintext envelope is signed, the digest value 292 used to create the signature is calculated over the binary value of 293 the payload data. That is, the value of the payload before the 294 encoding (Base-64, JSON-B) is applied. 296 1.2.2. Signing Individual Encrypted Envelopes 298 When an individual plaintext envelope is signed, the digest value 299 used to create the signature is calculated over the binary value of 300 the payload data. That is, the value of the payload after encryption 301 but before the encoding (Base-64, JSON-B) is applied. 303 Use of signing and encryption in combination presents the risk of 304 subtle attacks depending on the order in which signing and encryption 305 take place [Davis2001]. 307 Na?ve approaches in which an envelope is encrypted and then signed 308 present the possibility of a surreptitious forwarding attack. For 309 example, Alice signs an envelope and sends it to Mallet who then 310 strips off Alice's signature and sends the envelope to Bob. 312 Na?ve approaches in which an envelope is signed and then encrypted 313 present the possibility of an attacker claiming authorship of a 314 ciphertext. For example, Alice encrypts a ciphertext for Bob and 315 then signs it. Mallet then intercepts the envelope and sends it to 316 Bob. 318 While neither attack is a concern in all applications, both attacks 319 pose potential hazards for the unwary and require close inspection of 320 application protocol design to avoid exploitation. 322 To prevent these attacks, each signature on an envelope that is 323 signed and encrypted MUST include a witness value that is calculated 324 by applying a MAC function to the signature value as described in 325 section XXX. 327 1.2.3. Signing sequences of envelopes 329 To sign multiple envelopes with a single signature, we first 330 construct a Merkle tree of the envelope payload digest values and 331 then sign the root of the Merkle tree. 333 [This is not yet implemented but will be soon.] 335 1.3. Container 337 DARE Container is a message and file syntax that allows a sequence of 338 data frames to be represented with cryptographic integrity, 339 signature, and encryption enhancements to be constructed in an append 340 only format. 342 The format is designed to meet the requirements of a wide range of 343 use cases including: 345 * Recording transactions in persistent storage. 347 * Synchronizing transaction logs between hosts. 349 * File archive. 351 * Message spool. 353 * Signing and encrypting single data items. 355 * Incremental encryption and authentication of server logs. 357 1.3.1. Container Format 359 A Container consists of a sequence of variable length Frames. Each 360 frame consists of a forward length indicator, the framed data and a 361 reverse length indicator. The reverse length indicator is written 362 out backwards allowing the length and thus the frame to be read in 363 the reverse direction: 365 (Artwork only available as svg: No external link available, see 366 draft-hallambaker-mesh-dare-06.html for artwork.) 368 Figure 5 370 Each frame contains a single DARE Envelope consisting of a Header, 371 Payload and Trailer (if required). The first frame in a container 372 describes the container format options and defaults. These include 373 the range of encoding options for frame metadata supported and the 374 container profiles to which the container conforms. 376 All internal data formats support use of pointers of up to 64 bits 377 allowing containers of up to 18 exabytes to be written. 379 Five container types are currently specified: 381 Simple The container does not provide any index or content integrity 382 checks. 384 Tree Frame headers contain entries that specify the start position 385 of previous frames at the apex of the immediately enclosing binary 386 tree. This enables efficient random access to any frame in the 387 file. 389 Digest Each frame trailer contains a "PayloadDigest" field. 390 Modification of the payload will cause verification of the 391 "PayloadDigest" value to fail on that frame. 393 Chain Each frame trailer contains "PayloadDigest" and "ChainDigest" 394 fields allowing modifications to the payload data to be detected. 395 Modification of the payload will cause verification of the 396 "PayloadDigest" value to fail on that frame and verification of 397 the "ChainDigest" value to fail on all subsequent frames. 399 Merkle Tree Frame headers contain entries that specify the start 400 position of previous frames at the apex of the immediately 401 enclosing binary tree. Frame Trailers contain TreeDigestPartial 402 and TreeDigestFinal entries forming a Merkle digest tree. 404 1.3.2. Write 406 In normal circumstances, Containers are written as an append only 407 log. As with Envelopes, integrity information (payload digest, 408 signatures) is written to the entry trailer. Thus, large payloads 409 may be written without the need to buffer the payload data _provided 410 that_ the content length is known in advance. 412 Should exceptional circumstances require, Container entries MAY be 413 erased by overwriting the Payload and/or parts of the Header content 414 without compromising the ability to verify other entries in the 415 container. If the entry Payload is encrypted, it is sufficient to 416 erase the container salt value to render the container entry 417 effectively inaccessible (though recovery might still be possible if 418 the original salt value can be recovered from the storage media. 420 1.3.3. Encryption and Authentication 422 Frame payloads and associated attributes MAY be encrypted and/or 423 authenticated in the same manner as Envelopes. 425 _Incremental encryption_ is supported allowing encryption parameters 426 from a single public key exchange operation to be applied to encrypt 427 multiple frames. The public key exchange information is specified in 428 the first encrypted frame and subsequent frames encrypted under those 429 parameters specify the location at which the key exchange information 430 is to be found by means of the ExchangePosition field which MUST 431 specify a location that is earlier in the file. 433 To avoid cryptographic vulnerabilities resulting from key re-use, the 434 DARE key exchange requires that each encrypted sequence use an 435 encryption key and initialization vector derived from the master key 436 established in the public key exchange by means of a unique salt. 438 Each Envelope and by extension, each Container frame MUST specify a 439 unique salt value of at least 128 bits. Since the encryption key is 440 derived from the salt value by means of a Key Derivation Function, 441 erasure of the salt MAY be used as a means of rendering the payload 442 plaintext value inaccessible without changing the payload value. 444 1.3.4. Integrity and Signature 446 Signatures MAY be applied to a payload digest, the final digest in a 447 chain or tree. The chain and tree digest modes allow a single 448 signature to be used to authenticate all frame payloads in a 449 container. 451 The tree signature mode is particularly suited to applications such 452 as file archives as it allows files to be verified individually 453 without requiring the signer to sign each individually. Furthermore, 454 in applications such as code signing, it allows a single signature to 455 be used to verify both the integrity of the code and its membership 456 of the distribution. 458 As with DARE Envelope, the signature mechanism does not specify the 459 interpretation of the signature semantics. The presence of a 460 signature demonstrates that the holder of the private key applied it 461 to the specified digest value but not their motive for doing so. 462 Describing such semantics is beyond the scope of this document and is 463 deferred to future work. 465 1.3.5. Redaction 467 The chief disadvantage of using an append-only format is that 468 containers only increase in size. In many applications, much of the 469 data in the container becomes redundant or obsolete and a process 470 analogous to garbage collection is required. This process is called 471 _redaction_. 473 The simplest method of redaction is to create a new container and 474 sequentially copy each entry from the old container to the new, 475 discarding redundant frames and obsolete header information. 477 For example, partial index records may be consolidated into a single 478 index record placed in the last frame of the container. Unnecessary 479 signature and integrity data may be discarded and so on. 481 While redaction could in principle be effected by moving data in- 482 place in the existing container, supporting this approach in a robust 483 fashion is considerably more complex as it requires backward 484 references in subsequent frames to be overridden as each frame is 485 moved. 487 1.3.6. Alternative approaches 489 Many file proprietary formats are in use that support some or all of 490 these capabilities but only a handful have public, let alone open, 491 standards. DARE Container is designed to provide a superset of the 492 capabilities of existing message and file syntaxes, including: 494 * Cryptographic Message Syntax [RFC5652] defines a syntax used to 495 digitally sign, digest, authenticate, or encrypt arbitrary message 496 content. 498 * The.ZIP File Format specification [ZIPFILE] developed by Phil 499 Katz. 501 * The BitCoin Block chain [BLOCKCHAIN]. 503 * JSON Web Encryption and JSON Web Signature 505 Attempting to make use of these specifications in a layered fashion 506 would require at least three separate encoders and introduce 507 unnecessary complexity. Furthermore, there is considerable overlap 508 between the specifications providing multiple means of achieving the 509 same ends, all of which must be supported if decoders are to work 510 reliably. 512 1.3.7. Efficiency 514 Every data format represents a compromise between different concerns, 515 in particular: 517 Compactness The space required to record data in the encoding. 519 Memory Overhead The additional volatile storage (RAM) required to 520 maintain indexes etc. to support efficient retrieval operations. 522 Number of Operations The number of operations required to retrieve 523 data from or append data to an existing encoded sequence. 525 Number of Disk Seek Operations Optimizing the response time of 526 magnetic storage media to random access read requests has 527 traditionally been one of the central concerns of database design. 529 The DARE Container format is designed to the assumption that this 530 will cease to be a concern as solid state media replaces magnetic. 532 While the cost of storage of all types has declined rapidly over the 533 past decades, so has the amount of data to be stored. DARE Container 534 represents a pragmatic balance of these considerations for current 535 technology. In particular, since payload volumes are likely to be 536 very large, memory and operational efficiency are considered higher 537 priorities than compactness. 539 2. Definitions 541 2.1. Related Specifications 543 The DARE Envelope and Container formats are based on the following 544 existing standards and specifications. 546 Object serialization The JSON-B [draft-hallambaker-jsonbcd] encoding 547 is used for object serialization. This encoding is an extension 548 of the JavaScript Object Notation (JSON) [RFC7159]. 550 Message syntax The cryptographic processing model is based on JSON 551 Web Signature (JWS) [RFC7515], JSON Web Encryption (JWE) [RFC7516] 552 and JSON Web Key (JWK) [RFC7517]. 554 Cryptographic primitives. The HMAC-based Extract-and-Expand Key 555 Derivation Function [RFC5869] and Advanced Encryption Standard 556 (AES) Key Wrap with Padding Algorithm [RFC3394] are used. 558 The Uniform Data Fingerprint method of presenting data digests is 559 used for key identifiers and other purposes 560 [draft-hallambaker-mesh-udf]. 562 Cryptographic algorithms The cryptographic algorithms and 563 identifiers described in JSON Web Algorithms (JWA) [RFC7518] are 564 used together with additional algorithms as defined in the JSON 565 Object Signing and Encryption IANA registry [IANAJOSE]. 567 2.2. Requirements Language 569 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 570 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 571 document are to be interpreted as described in [RFC2119]. 573 2.3. Defined terms 575 The terms "Authentication Tag", "Content Encryption Key", "Key 576 Management Mode", "Key Encryption", "Direct Key Agreement", "Key 577 Agreement with Key Wrapping" and "Direct Encryption" are defined in 578 the JWE specification [RFC7516]. 580 The terms "Authentication", "Ciphertext", "Digital Signature", 581 "Encryption", "Initialization Vector (IV)", "Message Authentication 582 Code (MAC)", "Plaintext" and "Salt" are defined by the Internet 583 Security Glossary, Version 2 [RFC4949]. 585 Annotated Envelope A DARE Envelope that contains an "Annotations" 586 field with at least one entry. 588 Authentication Data A Message Authentication Code or authentication 589 tag. 591 Complete Envelope A DARE envelope that contains the key exchange 592 information necessary for the intended recipient(s) to decrypt it. 594 Detached Envelope A DARE envelope that does not contain the key 595 exchange information necessary for the intended recipient(s) to 596 decrypt it. 598 Encryption Context The master key, encryption algorithms and 599 associated parameters used to generate a set of one or more 600 enhanced data sequences. 602 Encoded data sequence (EDS) A sequence consisting of a salt, content 603 data and authentication data (if required by the encryption 604 context). 606 Enhancement Applying a cryptographic operation to a data sequence. 607 This includes encryption, authentication and both at the same 608 time. 610 Generator The party that generates a DARE envelope. 612 Group Encryption Key A key used to encrypt data to be read by a 613 group of users. This is typically achieved by means of some form 614 of proxy re-encryption or distributed key generation. 616 Group Encryption Key Identifier A key identifier for a group 617 encryption key. 619 Master Key (MK) The master secret from which keys are derived for 620 authenticating enhanced data sequences. 622 Recipient Any party that receives and processes at least some part 623 of a DARE envelope. 625 Related Envelope A set of DARE envelopes that share the same key 626 exchange information and hence the same Master Key. 628 Uniform Data Fingerprint (UDF) The means of presenting the result of 629 a cryptographic digest function over a data sequence and content 630 type identifier specified in the Uniform Data Fingerprint 631 specification [draft-hallambaker-mesh-udf] 633 3. DARE Envelope Architecture 635 A DARE Envelope is a sequence of three parts: 637 Header A JSON object containing information a reader requires to 638 begin processing the envelope. 640 Payload An array of octets. 642 Trailer A JSON object containing information calculated from the 643 envelope payload. 645 For example, the following sequence is a JSON encoded Envelope with 646 an empty header, a payload of zero length and an empty trailer: 648 [ {}, "", {} ] 650 DARE Envelopes MAY be encoded using JSON serialization or a binary 651 serialization for greater efficiency. 653 JSON [RFC7159] Offers compatibility with applications and libraries 654 that support JSON. Payload data is encoded using Base64 incurring 655 a 33% overhead. 657 JSON-B [draft-hallambaker-jsonbcd] A superset of JSON encoding that 658 permits binary data to be encoded as a sequence of length-data 659 segments. This avoids the Base64 overhead incurred by JSON 660 encoding. Since JSON-B is a superset of JSON encoding, an 661 application can use a single decoder for either format. 663 JSON-C [draft-hallambaker-jsonbcd] A superset of JSON-C which 664 provides additional efficiency by allowing field tags and other 665 repeated string data to be encoded by reference to a dictionary. 666 Since JSON-C is a superset of JSON and JSON-B encodings, an 667 application can use a single decoder for all three formats. 669 DARE Envelope processors MUST support JSON serialization and SHOULD 670 support JSON-B serialization. 672 3.1. Processing Considerations 674 The DARE Envelope Syntax supports single pass encoding and decoding 675 without buffering of data. All the information required to begin 676 processing a DARE envelope (key agreement information, digest 677 algorithms), is provided in the envelope header. All the information 678 that is derived from envelope processing (authentication codes, 679 digest values, signatures) is presented in the envelope trailer. 681 The choice of envelope encoding does not affect the semantics of 682 envelope processing. A DARE Envelope MAY be reserialized under the 683 same serialization or converted from any of the specified 684 serialization to any other serialization without changing the 685 semantics or integrity properties of the envelope. 687 3.2. Content Metadata and Annotations 689 A header MAY contain header fields describing the payload content. 690 These include: 692 ContentType Specifies the IANA Media Type [RFC6838]. 694 Annotations A list of Encoded Data Sequences that provide 695 application specific annotations to the envelope. 697 For example, consider the following mail message: 699 From: Alice@example.com 700 To: bob@example.com 701 Subject: TOP-SECRET Product Launch Today! 703 The CEO told me the product launch is today. Tell no-one! 705 Existing encryption approaches require that header fields such as the 706 subject line be encrypted with the body of the message or not 707 encrypted at all. Neither approach is satisfactory. In this 708 example, the subject line gives away important information that the 709 sender probably assumed would be encrypted. But if the subject line 710 is encrypted together with the message body, a mail client must 711 retrieve at least part of the message body to provide a 'folder' 712 view. 714 The plaintext form of the equivalent DARE Message encoding is: 716 [{ 717 "Annotations":["iAEBiBdGcm9tOiBBbGljZUBleGFtcGxlLmNvbYgA", 718 "iAECiBNUbzogYm9iQGV4YW1wbGUuY29tiAA", 719 "iAEDiClTdWJqZWN0OiBUT1AtU0VDUkVUIFByb2R1Y3QgTGF1bmNoIFRvZGF5 720 IYgA" 721 ], 722 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS1t 723 YWlsIn0"}, 724 "VGhlIENFTyB0b2xkIG1lIHRoZSBwcm9kdWN0IGxhdW5jaCBpcyB0b2RheS4gVGVs 725 bCBuby1vbmUh" 726 ] 728 This contains the same information as before but the mail message 729 headers are now presented as a list of Encoded Data Sequences. 731 3.3. Encoded Data Sequence 733 An encoded data sequence (EDS) is a sequence of octets that encodes a 734 data sequence according to cryptographic enhancements specified in 735 the context in which it is presented. An EDS MAY be encrypted and 736 MAY be authenticated by means of a MAC. The keys and other 737 cryptographic parameters used to apply these enhancements are derived 738 from the cryptographic context and a Salt prefix specified in the EDS 739 itself. 741 An EDS sequence contains exactly three binary fields encoded in 742 JSON-B serialization as follows: 744 Salt Prefix A sequence of octets used to derive the encryption key, 745 Initialization Vector and MAC key as required. 747 Body The plaintext or encrypted content. 749 Authentication Tag The authentication code value in the case that 750 the cryptographic context specifies use of authenticated 751 encryption or a MAC, otherwise is a zero-length field. 753 Requiring all three fields to be present, even in cases where they 754 are unnecessary simplifies processing at the cost of up to six 755 additional data bytes. 757 The encoding of the 'From' header of the previous example as a 758 plaintext EDS is as follows: 760 88 01 761 01 762 88 17 763 46 72 6f 6d 3a 20 41 6c 69 63 65 40 65 78 61 6d 764 70 6c 65 2e 63 6f 6d 765 88 00 767 3.4. Encryption and Integrity 769 Encryption and integrity protections MAY be applied to any DARE 770 Envelope Payload and Annotations. 772 The following is an encrypted version of the message shown earlier. 773 The payload and annotations have both increased in size as a result 774 of the block cipher padding. The header now includes Recipients and 775 Salt fields to enable the content to be decoded. 777 [{ 778 "enc":"A256CBC", 779 "Salt":"-Xf8VWwwviL9kGw_R9HZvg", 780 "Annotations":["iAEBiCBjUwS_tAjj03k0D35IcJ-H2LV7_JRoUhGiIVUmH6Ng 781 9Q", 782 "iAECiCBMfcp7ZgfQ5XvvyOA_mkP38vHlL7JEPQBad1sdLY0qJA", 783 "iAEDiDCjd4_oDTtGluViAPb9RLg80TZA10Ld3WWCiEdGiVK14Cm_qweeQpL- 784 Ur7Vq3Ay9FU" 785 ], 786 "recipients":[{ 787 "kid":"MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 788 "epk":{ 789 "PublicKeyECDH":{ 790 "crv":"Ed25519", 791 "Public":"AFFO7udHlhaV-LMpJ4B-1ETJksN1RGm5qlARDYGGA6g"}}, 792 "wmk":"XqWXnwInu5g3a0MWjIubre4oJfGm069LGBETbUiVLK88GQvOlgej 793 Yg"} 794 ], 795 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS1t 796 YWlsIn0"}, 797 "olOsbrzB4-mRMf_LJlPk9GemBwKpq3h02-9BL0viQp3EIqLPGYNyMn11xvqLYdBv 798 1GRHEPVj_j7QuD3kauhrIQ" 799 ] 801 3.4.1. Key Exchange 803 The DARE key exchange is based on the JWE key exchange except that 804 encryption modes are intentionally limited and the output of the key 805 exchange is the DARE Master Key rather than the Content Encryption 806 Key. 808 A DARE Key Exchange MAY contain any number of Recipient entries, each 809 providing a means of decrypting the Master Key using a different 810 private key. 812 If the Key Exchange mechanism supports message recovery, Direct Key 813 Agreement is used, in all other cases, Key Wrapping is used. 815 This approach allows envelopes with one intended recipient to be 816 handled in the exact same fashion as envelopes with multiple 817 recipients. While this does require an additional key wrapping 818 operation, that could be avoided if an envelope has exactly one 819 intended recipient, this is offset by the reduction in code 820 complexity. 822 If the key exchange algorithm does not support message recovery (e.g. 823 Diffie Hellman and Elliptic Curve Diffie-Hellman), the HKDF Extract- 824 and-Expand Key Derivation Function is used to derive a master key 825 using the following info tag: 827 "dare-master" [64 61 72 65 2d 6d 61 73 74 65 72] Key derivation info 828 field used when deriving a master key from the output of a key 829 exchange. 831 The master key length is the maximum of the key size of the 832 encryption algorithm specified by the key exchange header, the key 833 size of the MAC algorithm specified by the key exchange header (if 834 used) and 256. 836 3.4.2. Key Identifiers 838 The JWE/JWS specifications define a kid field for use as a key 839 identifier but not how the identifier itself is constructed. All 840 DARE key identifiers are either UDF key fingerprints 841 [draft-hallambaker-mesh-udf] or Mesh/Recrypt Group Key Identifiers. 843 A UDF fingerprint is formed as the digest of an IANA content type and 844 the digested data. A UDF key fingerprint is formed with the content 845 type "application/pkix-keyinfo" and the digested data is the ASN.1 846 DER encoded PKIX certificate "keyInfo" sequence for the corresponding 847 public key. 849 A Group Key Identifier has the form @. Where 850 is a UDF key fingerprint and is the DNS 851 address of a service that provides the encryption service to support 852 decryption by group members. 854 3.4.3. Salt Derivation 856 A Master Salt is a sequence of 16 or more octets that is specified in 857 the Salt field of the header. 859 The Master Salt is used to derive salt values for the envelope 860 payload and associated encoded data sequences as follows. 862 Payload Salt = Master Salt 864 EDS Salt = Concatenate (Payload Salt Prefix, Master Salt) 866 Encoders SHOULD NOT generate salt values that exceed 1024 octets. 868 The salt value is opaque to the DARE encoding but MAY be used to 869 encode application specific semantics including: 871 * Frame number to allow reassembly of a data sequence split over a 872 sequence of envelopes which may be delivered out of order. 874 * Transmit the Master Key in the manner of a Kerberos ticket. 876 * Identify the Master Key under which the Enhanced Data Sequence was 877 generated. 879 * Enable access to the plaintext to be eliminated by erasure of the 880 encryption key. 882 For data erasure to be effective, the salt MUST be constructed so 883 that the difficulty of recovering the key is sufficiently high that 884 it is infeasible. For most purposes, a salt with 128 bits of 885 appropriately random data is sufficient. 887 3.4.4. Key Derivation 889 Encryption and/or authentication keys are derived from the Master Key 890 using a Extract-and-Expand Key Derivation Function as follows: 892 0. The Master Key and salt value are used to extract the PRK 893 (pseudorandom key) 895 1. The PRK is used to derive the algorithm keys using the 896 application specific information input for that key type. 898 The application specific information inputs are: 900 "dare-encrypt" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 901 encryption or encryption with authentication key. 903 "dare-iv" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 904 initialization vector. 906 "dare-mac" [dare-mac] To generate a Message Authentication Code key. 908 3.5. Signature 910 While encryption and integrity enhancements can be applied to any 911 part of a DARE Envelope, signatures are only applied to payload 912 digest values calculated over one or more envelope payloads. 914 The payload digest value for an envelope is calculated over the 915 binary payload data. That is, after any encryption enhancement has 916 been applied but before the envelope encoding is applied. This 917 allows envelopes to be converted from one encoding to another without 918 affecting signature verification. 920 Single Payload The signed value is the payload digest of the 921 envelope payload. 923 Multiple Payload. The signed value is the root of a Merkle Tree in 924 which the payload digest of the envelope is one of the leaves. 926 Verification of a multiple payload signature naturally requires the 927 additional digest values required to construct the Merkle Tree. 928 These are provided in the Trailer in a format that permits multiple 929 signers to reference the same tree data. 931 3.6. Algorithms 933 3.6.1. Field: kwd 935 The key wrapping and derivation algorithms. 937 Since the means of public key exchange is determined by the key 938 identifier of the recipient key, it is only necessary to specify the 939 algorithms used for key wrapping and derivation. 941 The default (and so far only) algorithm is kwd-aes-sha2-256-256. 943 Advanced Encryption Standard (AES) Key Wrap with Padding Algorithm 944 [RFC3394] is used to wrap the Master Exchange Key. AES 256 is used. 946 HMAC-based Extract-and-Expand Key Derivation Function [RFC5869] is 947 used for key derivation. SHA-2-256 is used for the hash function. 949 4. DARE Container Architecture 950 4.1. Container Navigation 952 Three means of locating frames in a container are supported: 954 Sequential Access frames sequentially starting from the start or the 955 end of the container. 957 Binary search Access any container frame by frame number in 958 O(log_(2)(n)) time by means of a binary tree constructed while the 959 container is written. 961 Index Access and container frame by frame number or by key by means 962 of an index record. 964 All DARE Containers support sequential access. Only tree and Merkle 965 tree containers support binary search access. An index frame MAY be 966 written appended to any container and provides O(1) access to any 967 frame listed in the index. 969 Two modes of compilation are considered: 971 Monolithic Frames are added to the container in a single operation, 972 e.g. file archives, 974 Incremental Additional frames are written to the container at 975 various intervals after it was originally created, e.g. server 976 logs, message spools. 978 In the monolithic mode, navigation requirements are best met by 979 writing an index frame to the end of the container when it is 980 complete. It is not necessary to construct a binary search tree 981 unless a Merkle tree integrity check is required. 983 In the incremental mode, Binary search provides an efficient means of 984 locating frames by frame number but not by key. Writing a complete 985 index to the container every _m_ write operations provides _O(m)_ 986 search access but requires O(n^(2)) storage. 988 Use of partial indexes provides a better compromise between speed and 989 efficiency. A partial index is written out every _m_ frames where 990 _m_ is a power of two. A complete index is written every time a 991 binary tree apex record is written. This approach provides for 992 O(log_(2)(n)) search with incremental compilation with approximately 993 double the overhead of the monolithic case. 995 4.1.1. Tree 997 Binary search is supported by means of the "TreePosition" parameter 998 specified in the FrameHeader. This parameter specifies the value of 999 the immediately preceding apex. 1001 Calculation of the immediately preceding apex is most easily 1002 described by representing the array index in binary with base of 1 1003 (rather than 0). An array index that is a power of 2 (2, 4, 8, 16, 1004 etc.) will be the apex of a complete tree. Every other array index 1005 has the value of the sum of a set of powers of 2 and the immediately 1006 preceding apex will be the value of the next smallest power of 2 in 1007 the sum. 1009 For example, to find the immediately preceding apex for frame 5, we 1010 add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame 1011 is 4. 1013 The values of Tree Position are shown for the first 8 frames in 1014 figure xx below: 1016 (Artwork only available as svg: No external link available, see 1017 draft-hallambaker-mesh-dare-06.html for artwork.) 1019 Figure 6 1021 An algorithm for efficiently calculating the immediately preceding 1022 apex is provided in Appendix C. 1024 4.1.2. Position Index 1026 Contains a table of frame number, position pairs pointing to prior 1027 locations in the file. 1029 4.1.3. Metadata Index 1031 Contains a list of IndexMeta entries. Each entry contains a metadata 1032 description and a list of frame indexes (not positions) of frames 1033 that match the description. 1035 4.2. Integrity Mechanisms 1037 Frame sequences in a DARE container MAY be protected against a frame 1038 insertion attack by means of a digest chain, a binary Merkle tree or 1039 both. 1041 4.2.1. Digest Chain calculation 1043 A digest chain is simple to implement but can only be verified if the 1044 full chain of values is known. Appending a frame to the chain has 1045 _O(1)_ complexity but verification has _O(n)_ complexity: 1047 (Artwork only available as svg: No external link available, see 1048 draft-hallambaker-mesh-dare-06.html for artwork.) 1050 Figure 7 1052 The value of the chain digest for the first frame (frame 0) is 1053 _H(H(null)+H(Payload_(0))), where null is a zero length octet 1054 sequence and payloadn is the sequence of payload data bytes for frame 1055 n_ 1057 The value of the chain digest for frame _n is H(H(Payload_(n-1) 1058 +H(Payloadn)), where A+B stands for concatenation of the byte 1059 sequences A and B._ 1061 4.2.2. Binary Merkle tree calculation 1063 The tree index mechanism describe earlier may be used to implement a 1064 binary Merkle tree. The value TreeDigest specifies the apex value of 1065 the tree for that node. 1067 Appending a frame to the chain has O(log_(2) (n)) complexity provided 1068 that the container format supports at least the binary tree index. 1069 Verifying a chain has O(log_(2) (n)) complexity, provided that the 1070 set of necessary digest inputs is known. 1072 To calculate the value of the tree digest for a node, we first 1073 calculate the values of all the sub trees that have their apex at 1074 that node and then calculate the digest of that value and the 1075 immediately preceding local apex. 1077 4.2.3. Signature 1079 Payload data MAY be signed using a JWS [RFC7515] as applied in the 1080 Envelope. 1082 Signatures are specified by the "Signatures" parameter in the content 1083 header. The data that the signature is calculated over is defined by 1084 the typ parameter of the Signature as follows. 1086 "Payload" The value of the "PayloadDigest" parameter 1088 "Chain" The value of the "ChainDigest" parameter 1089 "Tree" The value of the "TreeDigestFinal" parameter 1091 If the "typ" parameter is absent, the value Payload is implied. 1093 A frame MAY contain multiple signatures created with the same signing 1094 key and different typ values. 1096 The use of signatures over chain and tree digest values permit 1097 multiple frames to be validated using a single signature verification 1098 operation. 1100 5. DARE Schema 1102 A DARE Envelope consists of a Header, an Enhanced Data Sequence (EDS) 1103 and an optional trailer. This section describes the JSON data fields 1104 used to construct headers, trailers and complete messages. 1106 Wherever possible, fields from JWE, JWS and JWK have been used. In 1107 these cases, the fields have the exact same semantics. Note however 1108 that the classes in which these fields are presented have different 1109 structure and nesting. 1111 5.1. Message Classes 1113 A DARE Message contains a single DAREMessageSequence in either the 1114 JSON or Compact serialization as directed by the protocol in which it 1115 is applied. 1117 5.1.1. Structure: DareEnvelopeSequence 1119 A DARE Message containing Header, EDS and Trailer in JSON object 1120 encoding. Since a DAREMessage is almost invariably presented in JSON 1121 sequence or compact encoding, use of the DAREMessage subclass is 1122 preferred. 1124 Although a DARE Message is functionally an object, it is serialized 1125 as an ordered sequence. This ensures that the message header field 1126 will always precede the body in a serialization, this allowing 1127 processing of the header information to be performed before the 1128 entire body has been received. 1130 Header: DareHeader (Optional) The message header. May specify the 1131 key exchange data, pre-signature or signature data, cloaked 1132 headers and/or encrypted data sequences. 1134 Body: Binary (Optional) The message body 1135 Trailer: DareTrailer (Optional) The message trailer. If present, 1136 this contains the signature. 1138 5.2. Header and Trailer Classes 1140 A DARE Message sequence MUST contain a (possibly empty) DAREHeader 1141 and MAY contain a DARETrailer. 1143 5.2.1. Structure: DareTrailer 1145 A DARE Message Trailer 1147 Signatures: DareSignature [0..Many] A list of signatures. A message 1148 trailer MUST NOT contain a signatures field if the header contains 1149 a signatures field. 1151 SignedData: Binary (Optional) Contains a DAREHeader object 1153 PayloadDigest: Binary (Optional) If present, contains the digest of 1154 the Payload. 1156 ChainDigest: Binary (Optional) If present, contains the digest of 1157 the PayloadDigest values of this frame and the frame immediately 1158 preceding. 1160 TreeDigest: Binary (Optional) If present, contains the Binary Merkle 1161 Tree digest value. 1163 5.2.2. Structure: DareHeader 1165 Inherits: DareTrailer 1167 A DARE Message Header. Since any field that is present in a trailer 1168 MAY be placed in a header instead, the message header inherits from 1169 the trailer. 1171 EnvelopeID: String (Optional) Unique identifier 1173 EncryptionAlgorithm: String (Optional) The encryption algorithm as 1174 specified in JWE 1176 DigestAlgorithm: String (Optional) Digest Algorithm. If specified, 1177 tells decoder that the digest algorithm is used to construct a 1178 signature over the message payload. 1180 Salt: Binary (Optional) Salt value used to derrive cryptographic 1181 parameters for the content data. 1183 Malt: Binary (Optional) Hash of the Salt value used to derrive 1184 cryptographic parameters for the content data. This field SHOULD 1185 NOT be present if the Salt field is present. It is used to allow 1186 the salt value to be erased (thus rendering the payload content 1187 irrecoverable) without affecting the ability to calculate the 1188 payload digest value. 1190 Cloaked: Binary (Optional) If present in a header or trailer, 1191 specifies an encrypted data block containing additional header 1192 fields whose values override those specified in the message and 1193 context headers. 1195 When specified in a header, a cloaked field MAY be used to conceal 1196 metadata (content type, compression) and/or to specify an 1197 additional layer of key exchange. That applies to both the 1198 Message body and to headers specified within the cloaked header. 1200 Processing of cloaked data is described in... 1202 EDSS: Binary [0..Many] If present, the Annotations field contains a 1203 sequence of Encrypted Data Segments encrypted under the message 1204 Master Key. The interpretation of these fields is application 1205 specific. 1207 Signers: DareSigner [0..Many] A list of 'presignature' 1209 Recipients: DareRecipient [0..Many] A list of recipient key exchange 1210 information blocks. 1212 ContentMetaData: Binary (Optional) If present contains a JSON 1213 encoded ContentInfo structure which specifies plaintext content 1214 metadata and forms one of the inputs to the envelope digest value. 1216 ContainerInfo: ContainerInfo (Optional) Information that describes 1217 container information 1219 ContainerIndex: ContainerIndex (Optional) An index of records in the 1220 current container up to but not including this one. 1222 5.2.3. Structure: ContentMeta 1224 UniqueID: String (Optional) Unique object identifier 1226 Labels: String [0..Many] List of labels that are applied to the 1227 payload of the frame. 1229 KeyValues: KeyValue [0..Many] List of key/value pairs describing the 1230 payload of the frame. 1232 MessageType: String (Optional) The mesh message type 1234 ContentType: String (Optional) The content type field as specified 1235 in JWE 1237 Paths: String [0..Many] List of filename paths for the payload of 1238 the frame. 1240 Filename: String (Optional) The original filename under which the 1241 data was stored. 1243 Event: String (Optional) Operation on the header 1245 Created: DateTime (Optional) Initial creation date. 1247 Modified: DateTime (Optional) Date of last modification. 1249 First: Integer (Optional) Frame number of the first object instance 1250 value. 1252 Previous: Integer (Optional) Frame number of the immediately prior 1253 object instance value 1255 5.3. Cryptographic Data 1257 DARE Message uses the same fields as JWE and JWS but with different 1258 structure. In particular, DARE messages MAY have multiple recipients 1259 and multiple signers. 1261 5.3.1. Structure: DareSigner 1263 The signature value 1265 Dig: String (Optional) Digest algorithm hint. Specifying the digest 1266 algorithm to be applied to the message body allows the body to be 1267 processed in streaming mode. 1269 Alg: String (Optional) Key exchange algorithm 1271 KeyIdentifier: String (Optional) Key identifier of the signature 1272 key. 1274 Certificate: X509Certificate (Optional) PKIX certificate of signer. 1276 Path: X509Certificate (Optional) PKIX certificates that establish a 1277 trust path for the signer. 1279 5.3.2. Structure: X509Certificate 1281 X5u: String (Optional) URL identifying an X.509 public key 1282 certificate 1284 X5: Binary (Optional) An X.509 public key certificate 1286 5.3.3. Structure: DareSignature 1288 Inherits: DareSigner 1290 The signature value 1292 Manifest: Binary (Optional) The data description that was signed. 1294 SignatureValue: Binary (Optional) The signature value as an Enhanced 1295 Data Sequence under the message Master Key. 1297 WitnessValue: Binary (Optional) The signature witness value used on 1298 an encrypted message to demonstrate that the signature was 1299 authorized by a party with actual knowledge of the encryption key 1300 used to encrypt the message. 1302 5.3.4. Structure: DareRecipient 1304 Recipient information 1306 KeyIdentifier: String (Optional) Key identifier for the encryption 1307 key. 1309 The Key identifier MUST be either a UDF fingerprint of a key or a 1310 Group Key Identifier 1312 KeyWrapDerivation: String (Optional) The key wrapping and derivation 1313 algorithms. 1315 WrappedMasterKey: Binary (Optional) The wrapped master key. The 1316 master key is encrypted under the result of the key exchange. 1318 RecipientKeyData: String (Optional) The per-recipient key exchange 1319 data. 1321 6. DARE Container Schema 1323 TBS stuff 1325 6.1. Container Headers 1327 TBS stuff 1329 6.1.1. Structure: ContainerInfo 1331 DataEncoding: String (Optional) Specifies the data encoding for the 1332 header section of for the following frames. This value is ONLY 1333 valid in Frame 0 which MUST have a header encoded in JSON. 1334 Information that describes container information 1336 Index: Integer (Optional) The record index within the file. This 1337 MUST be unique and satisfy any additional requirements determined 1338 by the ContainerType. 1340 ContainerType: String (Optional) Specifies the container type for 1341 the following records. 1343 IsMeta: Boolean (Optional) If true, the current frame is a meta 1344 frame and does not contain a payload. 1346 Note: Meta frames MAY be present in any container. Applications 1347 MUST accept containers that contain meta frames at any position in 1348 the file. Applications MUST NOT interpret a meta frame as a data 1349 frame with an enpty payload. 1351 Default: Boolean (Optional) If set true in a persistent container, 1352 specifies that this record contains the default object for the 1353 container. 1355 TreePosition: Integer (Optional) Position of the frame containing 1356 the apex of the preceding sub-tree. 1358 IndexPosition: Integer (Optional) Specifies the position in the file 1359 at which the last index entry is to be found 1361 ExchangePosition: Integer (Optional) Specifies the position in the 1362 file at which the key exchange data is to be found 1364 6.2. Index Structures 1366 TBS stuff 1368 6.2.1. Structure: ContainerIndex 1370 A container index 1371 Full: Boolean (Optional) If true, the index is complete and contains 1372 position entries for all the frames in the file. If absent or 1373 false, the index is incremental and only contains position entries 1374 for records added since the last frame containing a 1375 ContainerIndex. 1377 Positions: IndexPosition [0..Many] List of container position 1378 entries 1380 Metas: IndexMeta [0..Many] List of container position entries 1382 6.2.2. Structure: IndexPosition 1384 Specifies the position in a file at which a specified record index is 1385 found 1387 Index: Integer (Optional) The record index within the file. 1389 Position: Integer (Optional) The record position within the file 1390 relative to the index base. 1392 6.2.3. Structure: KeyValue 1394 Specifies a key/value entry 1396 Key: String (Optional) The key 1398 Value: String (Optional) The value corresponding to the key 1400 6.2.4. Structure: IndexMeta 1402 Specifies the list of index entries at which a record with the 1403 specified metadata occurrs. 1405 Index: Integer [0..Many] List of record indicies within the file 1406 where frames matching the specified criteria are found. 1408 ContentType: String (Optional) Content type parameter 1410 Paths: String [0..Many] List of filename paths for the current 1411 frame. 1413 Labels: String [0..Many] List of labels that are applied to the 1414 current frame. 1416 7. Dare Container Applications 1418 DARE Containers are used to implement two forms of persistence store 1419 to support Mesh operations: 1421 Catalogs A set of related items which MAY be added, modified or 1422 deleted at any time. 1424 Spools A list of related items whose status MAY be changed at any 1425 time but which are immutable once added. 1427 Since DARE Containers are an append only log format, entries can only 1428 be modified or deleted by adding items to the log to change the 1429 status of previous entries. It is always possible to undo any 1430 operation on a catalog or spool unless the underlying container is 1431 purged or the individual entries modified. 1433 7.1. Catalog 1435 Catalogs contain a set of entries, each of which is distinguished by 1436 a unique identifier. 1438 Three operations are supported: 1440 Add Addition of the entry to the catalog 1442 Update Modification of the data associated with the entry excluding 1443 the identifier 1445 Delete Removal of the entry from the catalog 1447 The set of valid state transitions is defined by the Finite State 1448 machine: 1450 (Add-Update*-Delete)* 1452 Catalogs are used to represent sets of persistent objects associated 1453 with a Mesh Service Account. The user's set of contacts for example. 1454 Each contact entry may be modified many times over time but refers to 1455 the same subject for its entire lifetime. 1457 SchemaCatalog 1459 7.2. Spool 1461 Spools contain lists of entries, each of which is distinguished by a 1462 unique identifier. 1464 Four operations are supported: 1466 Post Addition of the entry to the spool 1468 Processed Marks the entry as having been processed. 1470 Unprocessed Returns the entry to the unread state. 1472 Delete Mark the entry as deleted allowing recovery of associated 1473 storage in a subsequent purge operation. 1475 The set of valid state transitions is defined by the Finite State 1476 machine: 1478 Post-(Processed| Unprocessed| Delete *) 1480 Spools are used to represent time sequence ordered entries such as 1481 lists of messages being sent or received, task queues and transaction 1482 logs. 1484 SchemaCatalog 1486 7.3. Archive 1488 A DARE Archive is a DARE Container whose entries contain files. This 1489 affords the same functionality as a traditional ZIP or tar archive 1490 but with the added cryptographic capabilities provided by the DARE 1491 format. 1493 8. Future Work 1495 The current specification describes an approach in which containers 1496 are written according to a strict append-only policy. Greater 1497 flexibility may be achieved by loosening this requirement allowing 1498 record(s) at the end of the container to be overwritten. 1500 8.1. Terminal integrity check 1502 A major concern when operating a critical service is the possibility 1503 of a hardware or power failure occurring during a write operation 1504 causing the file update to be incomplete. While most modern 1505 operating systems have effective mechanisms in place to prevent 1506 corruption of the file system itself in such circumstances, this does 1507 not provide sufficient protection at the application level. 1509 Appending a null record containing a container-specific magic number 1510 provides an effective means of detecting this circumstance that can 1511 be quickly verified. 1513 If a container specifies a terminal integrity check value in the 1514 header of frame zero, the container is considered to be in an 1515 incomplete write state if the final frame is not a null record 1516 specifying the magic number. 1518 When appending new records to such containers, the old terminal 1519 integrity check record is overwritten by the data being added and a 1520 new integrity check record appended to the end. 1522 8.2. Terminal index record 1524 A writer can maintain a complete (or partial) index of the container 1525 in its final record without additional space overhead by overwriting 1526 the prior index on each update. 1528 8.3. Deferred indexing 1530 The task of updating terminal indexes may be deferred to a time when 1531 the machine is not busy. This improves responsiveness and may avoid 1532 the need to re-index containers receiving a sequence of updates. 1534 This approach may be supported by appending new entries to the end of 1535 the container in the usual fashion and maintaining a record of 1536 containers to be updated as a separate task. 1538 When updating the index on a container that has been updated in this 1539 fashion, the writer must ensure that no data is lost even if the 1540 process is interrupted. The use of guard records and other 1541 precautions against loss of state is advised. 1543 9. Security Considerations 1545 This section describes security considerations arising from the use 1546 of DARE in general applications. 1548 Additional security considerations for use of DARE in Mesh services 1549 and applications are described in the Mesh Security Considerations 1550 guide [draft-hallambaker-mesh-security]. 1552 9.1. Encryption/Signature nesting 1554 9.2. Side channel 1556 9.3. Salt reuse 1558 10. IANA Considerations 1560 11. Acknowledgements 1562 A list of people who have contributed to the design of the Mesh is 1563 presented in [draft-hallambaker-mesh-architecture]. 1565 The name Data At Rest Encryption was proposed by Melhi Abdulhayo?lu. 1567 12. Appendix A: DARE Envelope Examples and Test Vectors 1569 13. Test Examples 1571 In the following examples, Alice's encryption private key parameters 1572 are: 1574 { 1575 "PrivateKeyECDH":{ 1576 "crv":"Ed25519", 1577 "Private":"YgyvWwNuk34RqSHZvwmn6qgOvZlhIdoyzVNceex7k5w"}} 1579 Alice's signature private key parameters are: 1581 { 1582 "PrivateKeyECDH":{ 1583 "crv":"Ed25519", 1584 "Private":"JWm8_i4mFf8R6cx6pUfVnuMuGab0ZlkAOtQaYmlTdRk"}} 1586 The body of the test message is the UTF8 representation of the 1587 following string: 1589 "This is a test long enough to require multiple blocks" 1591 The EDS sequences, are the UTF8 representation of the following 1592 strings: 1594 "Subject: Message metadata should be encrypted" 1595 "2018-02-01" 1597 13.1. Plaintext Message 1599 A plaintext message without associated EDS sequences is an empty 1600 header followed by the message body: 1602 { 1603 "DareEnvelope":[{}, 1604 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1605 BibG9ja3M" 1606 ]} 1608 13.2. Plaintext Message with EDS 1610 If a plaintext message contains EDS sequences, these are also in 1611 plaintext: 1613 { 1614 "DareEnvelope":[{ 1615 "Annotations":["iAEBiC1TdWJqZWN0OiBNZXNzYWdlIG1ldGFkYXRhIHNob3 1616 VsZCBiZSBlbmNyeXB0ZWSIAA", 1617 "iAECiAoyMDE4LTAyLTAxiAA" 1618 ]}, 1619 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1620 BibG9ja3M" 1621 ]} 1623 13.3. Encrypted Message 1625 The creator generates a master session key: 1627 93 53 09 94 F4 C5 D1 17 B3 E1 3E 86 C8 17 FE 3C 1628 7C 5E A1 BC 12 9B 36 96 2B A3 99 F2 47 6C 74 86 1630 For each recipient of the message: 1632 The creator generates an ephemeral key: 1634 { 1635 "PrivateKeyECDH":{ 1636 "crv":"Ed25519", 1637 "Private":"oxs52uB94a1ooE3oNsQXWd3Wuk9l7CqZrf-e5MfgYS8"}} 1639 The key agreement value is calculated: 1641 7C 6D D6 B6 33 BB D9 CD CB C2 EA ED 0D 0D FA 57 1642 C2 AE 8F 60 C4 6E 5F 18 F4 97 7A 28 12 47 FA F7 1644 The key agreement value is used as the input to a HKDF key derivation 1645 function with the info parameter master to create the key used to 1646 wrap the master key: 1648 81 A3 17 09 D3 DB 11 55 EA C5 6F DA 21 86 EA 98 1649 E0 36 85 5B 54 FE 78 56 0D 9A 5F 2B 80 96 3D A3 1651 The wrapped master key is: 1653 BE 0A EF B8 8A 38 9A A9 56 3F 67 FD 3F CC AC E0 1654 A6 E8 98 49 57 99 7D 5D E2 E6 86 3B CA 12 65 90 1655 DA 1F 41 42 3E B5 84 2F 1657 This information is used to calculate the Recipient information shown 1658 in the example below. 1660 To encrypt a message, we first generate a unique salt value: 1662 36 41 DD 2F 7D 47 70 75 C6 EB 6C 45 76 04 11 6E 1664 The salt value and master key are used to generate the payload 1665 encryption key: 1667 DB D5 2E 6D 0E 4F D6 21 3D 50 40 97 C1 35 70 3A 1668 14 91 15 AB 72 6C 5F 74 45 0D 14 14 25 29 62 2D 1670 Since AES is a block cipher, we also require an initializarion 1671 vector: 1673 66 A4 0B 97 4C 37 DA C0 76 E6 C8 A5 B5 25 F8 55 1675 The output sequence is the encrypted bytes: 1677 96 3B 33 49 E4 40 89 45 59 1C 81 FF A4 B2 88 58 1678 B3 08 32 6E 1B CD A6 7A 05 FE 1A ED 64 29 0E 51 1679 55 03 FE B9 90 BF 36 A5 74 63 19 AD 76 C3 39 47 1680 69 F3 A5 D9 11 EB 33 28 EF A8 83 A5 25 CD 1D 58 1682 Since the message is not signed, there is no need for a trailer. The 1683 completed message is: 1685 { 1686 "DareEnvelope":[{ 1687 "enc":"A256CBC", 1688 "Salt":"NkHdL31HcHXG62xFdgQRbg", 1689 "recipients":[{ 1690 "kid":"MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 1691 "epk":{ 1692 "PublicKeyECDH":{ 1693 "crv":"Ed25519", 1694 "Public":"5dMMgpgHv7vsdwT1Q9fLDpjQTsG0WhEaJPNtyl-6HmM"} 1695 }, 1696 "wmk":"vgrvuIo4mqlWP2f9P8ys4KbomElXmX1d4uaGO8oSZZDaH0FCPr 1697 WELw"} 1698 ]}, 1699 "ljszSeRAiUVZHIH_pLKIWLMIMm4bzaZ6Bf4a7WQpDlFVA_65kL82pXRjGa12wz 1700 lHafOl2RHrMyjvqIOlJc0dWA" 1701 ]} 1703 13.4. Signed Message 1705 Signed messages specify the digest algorithm to be used in the header 1706 and the signature value in the trailer. Note that the digest 1707 algorithm is not optional since it serves as notice that a decoder 1708 should digest the payload value to enable signature verification. 1710 { 1711 "DareEnvelope":[{ 1712 "dig":"SHA2"}, 1713 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1714 BibG9ja3M", 1715 { 1716 "signatures":[{ 1717 "alg":"SHA2", 1718 "kid":"MC7T-C6CR-THZN-35QU-7GLC-GILJ-ZV7K", 1719 "signature":"f8fj6eOwbrLaSDgAmzW-qj95Z7DUkMkvrTEFlRXDbRgG 1720 BMjuVktELASgUlcD-_kdLEDMMOKKTuuJ2-sIiY06BA"} 1721 ], 1722 "PayloadDigest":"raim8SV5adPbWWn8FMM4mrRAQCO9A2jZ0NZAnFXWlG0x 1723 F6sWGJbnKSdtIJMmMU_hjarlIPEoY3vy9UdVlH5KAg"} 1724 ]} 1726 13.5. Signed and Encrypted Message 1728 A signed and encrypted message is encrypted and then signed. The 1729 signer proves knowledge of the payload plaintext by providing the 1730 plaintext witness value. 1732 { 1733 "DareEnvelope":[{ 1734 "enc":"A256CBC", 1735 "dig":"SHA2", 1736 "Salt":"kZlrg_NXLMIneamI7NIq0A", 1737 "recipients":[{ 1738 "kid":"MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 1739 "epk":{ 1740 "PublicKeyECDH":{ 1741 "crv":"Ed25519", 1742 "Public":"BtzafehndD3P0hXtZrDyulloJouPL-aAaejtVQoK4FY"} 1743 }, 1744 "wmk":"Ai5gfoA6l7fIDIQ9An5nfq87khaGs0dfvHxn2QqejqZb79iLKb 1745 3z3w"} 1746 ]}, 1747 "XjZFDcwG3N7M55BbU675s0gV32rraZKjz5khZDRPHROAgxmiP5YJzxArKgXOih 1748 iW90exJlH1QcDr4AKghm-hYA", 1749 { 1750 "signatures":[{ 1751 "alg":"SHA2", 1752 "kid":"MC7T-C6CR-THZN-35QU-7GLC-GILJ-ZV7K", 1753 "signature":"MqWR5bDbZ8_N01LlItVJ54UyIyaFcVu9Nh8OBX0cWq8O 1754 KSunykIWrPbYjCTNC7s1dcD2RoT_psx6PmL-EfAGDA", 1755 "witness":"euia295oDy60QLNrb3UVCB57R25J3OcCd9jZxM11B8Y"} 1756 ], 1757 "PayloadDigest":"obHGC-2CsriBsWNGrJNLt0zzQcY6GMrUah8K7xSXnnOC 1758 UfGv8GDtJQrhbxcaL3TELz6takJvnAAzmvAyFfovhQ"} 1759 ]} 1761 14. Appendix B: DARE Container Examples and Test Vectors 1763 The data payloads in all the following examples are identical, only 1764 the authentication and/or encryption is different. 1766 * Frame 1..n consists of 300 bytes being the byte sequence 00, 01, 1767 02, etc. repeating after 256 bytes. 1769 For conciseness, the raw data format is omitted for examples after 1770 the first, except where the data payload has been transformed, (i.e. 1771 encrypted). 1773 14.1. Simple container 1775 the following example shows a simple container with first frame and a 1776 single data frame: 1778 f4 82 1779 f0 7c 1780 f0 00 1781 f0 00 1782 82 f4 1783 f5 01 74 1784 f0 43 1785 f1 01 2c 1786 74 01 f5 1788 Since there is no integrity check, there is no need for trailer 1789 entries. The header values are: 1791 Frame 0 1793 { 1794 "ContentMetaData": "e30", 1795 "ContainerInfo": { 1796 "DataEncoding": "JSON", 1797 "Index": 0, 1798 "ContainerType": "List"}} 1800 [Empty trailer] 1802 Frame 1 1804 { 1805 "ContentMetaData": "e30", 1806 "ContainerInfo": { 1807 "Index": 1}} 1809 [Empty trailer] 1811 14.2. Payload and chain digests 1813 The following example shows a chain container with a first frame and 1814 three data frames. The headers of these frames is the same as before 1815 but the frames now have trailers specifying the PayloadDigest and 1816 ChainDigest values: 1818 Frame 0 1819 { 1820 "ContentMetaData": "e30", 1821 "ContainerInfo": { 1822 "DataEncoding": "JSON", 1823 "Index": 0, 1824 "ContainerType": "Chain"}} 1826 [Empty trailer] 1828 Frame 1 1830 { 1831 "ContentMetaData": "e30", 1832 "ContainerInfo": { 1833 "Index": 1}} 1835 { 1836 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1837 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1838 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1839 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1841 Frame 2 1843 { 1844 "ContentMetaData": "e30", 1845 "ContainerInfo": { 1846 "Index": 2}} 1848 { 1849 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1850 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1851 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1852 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1854 Frame 3 1856 { 1857 "ContentMetaData": "e30", 1858 "ContainerInfo": { 1859 "Index": 3}} 1861 { 1862 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1863 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1864 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1865 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1867 14.3. Merkle Tree 1869 The following example shows a chain container with a first frame and 1870 six data frames. The trailers now contain the TreePosition and 1871 TreeDigest values: 1873 Frame 0 1875 { 1876 "ContentMetaData": "e30", 1877 "ContainerInfo": { 1878 "DataEncoding": "JSON", 1879 "Index": 0, 1880 "ContainerType": "Merkle"}} 1882 [Empty trailer] 1884 Frame 1 1886 { 1887 "ContentMetaData": "e30", 1888 "ContainerInfo": { 1889 "Index": 1, 1890 "TreePosition": 0}} 1892 { 1893 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1894 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1895 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1896 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1898 Frame 2 1900 { 1901 "ContentMetaData": "e30", 1902 "ContainerInfo": { 1903 "Index": 2, 1904 "TreePosition": 360}} 1906 { 1907 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1908 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1909 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 1910 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 1912 Frame 3 1913 { 1914 "ContentMetaData": "e30", 1915 "ContainerInfo": { 1916 "Index": 3, 1917 "TreePosition": 360}} 1919 { 1920 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1921 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1922 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1923 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1925 Frame 4 1927 { 1928 "ContentMetaData": "e30", 1929 "ContainerInfo": { 1930 "Index": 4, 1931 "TreePosition": 1612}} 1933 { 1934 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1935 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1936 "TreeDigest": "vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvK 1937 Sj8gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"} 1939 Frame 5 1941 { 1942 "ContentMetaData": "e30", 1943 "ContainerInfo": { 1944 "Index": 5, 1945 "TreePosition": 1612}} 1947 { 1948 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1949 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1950 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1951 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1953 Frame 6 1954 { 1955 "ContentMetaData": "e30", 1956 "ContainerInfo": { 1957 "Index": 6, 1958 "TreePosition": 2867}} 1960 { 1961 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1962 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1963 "TreeDigest": "WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9S 1964 Z0mdr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"} 1966 14.4. Signed container 1968 The following example shows a tree container with a signature in the 1969 final record. The signing key parameters are: 1971 { 1972 "PrivateKeyECDH":{ 1973 "crv":"Ed25519", 1974 "Private":"JWm8_i4mFf8R6cx6pUfVnuMuGab0ZlkAOtQaYmlTdRk"}} 1976 The container headers and trailers are: 1978 Frame 0 1980 { 1981 "ContentMetaData": "e30", 1982 "ContainerInfo": { 1983 "DataEncoding": "JSON", 1984 "Index": 0, 1985 "ContainerType": "Merkle"}} 1987 [Empty trailer] 1989 Frame 1 1991 { 1992 "ContentMetaData": "e30", 1993 "ContainerInfo": { 1994 "Index": 1, 1995 "TreePosition": 0}} 1997 { 1998 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1999 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2000 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 2001 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 2003 Frame 2 2005 { 2006 "ContentMetaData": "e30", 2007 "ContainerInfo": { 2008 "Index": 2, 2009 "TreePosition": 360}} 2011 { 2012 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 2013 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2014 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 2015 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 2017 14.5. Encrypted container 2019 The following example shows a container in which all the frame 2020 payloads are encrypted under the same master secret established in a 2021 key agreement specified in the first frame. 2023 Frame 0 2025 { 2026 "enc": "A256CBC", 2027 "Salt": "R1Y3DxdC5H9efFRQdvGMSA", 2028 "recipients": [{ 2029 "kid": "MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 2030 "epk": { 2031 "PublicKeyECDH": { 2032 "crv": "Ed25519", 2033 "Public": "3PR_wusc8XRV44YD2-jXqrpYLXQp79a6-iLQWto2HOs"}}, 2034 "wmk": "KfF8MMItf9HmdtZU3t5bZUCc_5h5wT_pdSV3dVQg5UvoN8EuYNOXkg" 2035 }], 2036 "ContentMetaData": "e30", 2037 "ContainerInfo": { 2038 "DataEncoding": "JSON", 2039 "Index": 0, 2040 "ContainerType": "List"}} 2042 [Empty trailer] 2044 Frame 1 2045 { 2046 "enc": "A256CBC", 2047 "Salt": "BJih1YYYl-NA1tLYuVhbDQ", 2048 "ContentMetaData": "e30", 2049 "ContainerInfo": { 2050 "Index": 1}} 2052 [Empty trailer] 2054 Frame 2 2056 { 2057 "enc": "A256CBC", 2058 "Salt": "ikAqoKLde7dWZnYC5m8dbg", 2059 "ContentMetaData": "e30", 2060 "ContainerInfo": { 2061 "Index": 2}} 2063 [Empty trailer] 2065 Here are the container bytes. Note that the content is now encrypted 2066 and has expanded by 25 bytes. These are the salt (16 bytes), the AES 2067 padding (4 bytes) and the JSON-B framing (5 bytes). 2069 f5 01 e5 2070 f1 01 ce 2071 f0 10 2072 f0 00 2073 e5 01 f5 2074 f5 01 b0 2075 f0 7b 2076 f1 01 30 2077 b0 01 f5 2078 f5 01 b0 2079 f0 7b 2080 f1 01 30 2081 b0 01 f5 2083 The following example shows a container in which all the frame 2084 payloads are encrypted under separate key agreements specified in the 2085 payload frames. 2087 Frame 0 2088 { 2089 "ContentMetaData": "e30", 2090 "ContainerInfo": { 2091 "DataEncoding": "JSON", 2092 "Index": 0, 2093 "ContainerType": "List"}} 2095 [Empty trailer] 2097 Frame 1 2099 { 2100 "enc": "A256CBC", 2101 "Salt": "R2EXKJQEtN_afGwOWR22gw", 2102 "recipients": [{ 2103 "kid": "MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 2104 "epk": { 2105 "PublicKeyECDH": { 2106 "crv": "Ed25519", 2107 "Public": "s09EZ1rXTEumntSe3iuPLmzNsAD6Hg0B1Blch9c05HQ"}}, 2108 "wmk": "KHaQEptXjmk9A-sQeY0Rahc5SfPDCj_AFBWQCzApPI1Wt6zy3VTUCw" 2109 }], 2110 "ContentMetaData": "e30", 2111 "ContainerInfo": { 2112 "Index": 1}} 2114 [Empty trailer] 2116 Frame 2 2118 { 2119 "enc": "A256CBC", 2120 "Salt": "b0Cl1MCCpc2ZSMu4U1C3GA", 2121 "recipients": [{ 2122 "kid": "MDW4-OC5J-T6AW-BXGT-YCAZ-3VFB-B7MG", 2123 "epk": { 2124 "PublicKeyECDH": { 2125 "crv": "Ed25519", 2126 "Public": "NH799YFaKDv8UrYNwNs0NMreuX4R3ylcU5-8iZ0LorI"}}, 2127 "wmk": "JGyrAmtTgeQ4WGz-E04Ru7etlmZ7m7BMsXOckCVJobCKJojNfbnrOA" 2128 }], 2129 "ContentMetaData": "e30", 2130 "ContainerInfo": { 2131 "Index": 2}} 2133 [Empty trailer] 2135 15. Appendix C: Previous Frame Function 2137 public long PreviousFrame (long Frame) { 2138 long x2 = Frame + 1; 2139 long d = 1; 2141 while (x2 > 0) { 2142 if ((x2 & 1) == 1) { 2143 return x2 == 1 ? (d / 2) - 1 : Frame - d; 2144 } 2145 d = d * 2; 2146 x2 = x2 / 2; 2147 } 2148 return 0; 2149 } 2151 16. Appendix D: Outstanding Issues 2153 The following issues need to be addressed. 2155 +----------------+-----------------------------------------------+ 2156 | Issue | Description | 2157 +================+===============================================+ 2158 | X25519 | The examples currently use Edwards Curve25519 | 2159 | | for encryption. This should be Curve X25519 | 2160 +----------------+-----------------------------------------------+ 2161 | Indexing | No examples are given of indexing a container | 2162 +----------------+-----------------------------------------------+ 2163 | Archive | Should include a file archive example | 2164 +----------------+-----------------------------------------------+ 2165 | File Path | Mention the file path security issue in the | 2166 | | security considerations | 2167 +----------------+-----------------------------------------------+ 2168 | Security | Write Security considerations | 2169 | Considerations | | 2170 +----------------+-----------------------------------------------+ 2171 | AES-GCM | Switch to using AES GCM in the examples | 2172 +----------------+-----------------------------------------------+ 2173 | Witness | Complete handling of witness values. | 2174 +----------------+-----------------------------------------------+ 2175 | Schema | Complete the schema documentation | 2176 +----------------+-----------------------------------------------+ 2177 | Container Redo | Rework the container/header objects so that | 2178 | | these are separate classes and Header is an | 2179 | | entry in the Container header. | 2180 +----------------+-----------------------------------------------+ 2182 Table 1 2184 17. Normative References 2186 [draft-hallambaker-jsonbcd] 2187 Hallam-Baker, P., "Binary Encodings for JavaScript Object 2188 Notation: JSON-B, JSON-C, JSON-D", Work in Progress, 2189 Internet-Draft, draft-hallambaker-jsonbcd-15, 23 October 2190 2019, . 2193 [draft-hallambaker-mesh-architecture] 2194 Hallam-Baker, P., "Mathematical Mesh 3.0 Part I: 2195 Architecture Guide", Work in Progress, Internet-Draft, 2196 draft-hallambaker-mesh-architecture-12, 16 January 2020, 2197 . 2200 [draft-hallambaker-mesh-security] 2201 Hallam-Baker, P., "Mathematical Mesh 3.0 Part VII: 2202 Security Considerations", Work in Progress, Internet- 2203 Draft, draft-hallambaker-mesh-security-02, 23 October 2204 2019, . 2207 [draft-hallambaker-mesh-udf] 2208 Hallam-Baker, P., "Mathematical Mesh 3.0 Part II: Uniform 2209 Data Fingerprint.", Work in Progress, Internet-Draft, 2210 draft-hallambaker-mesh-udf-08, 6 January 2020, 2211 . 2214 [IANAJOSE] "[Reference Not Found!]". 2216 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2217 Requirement Levels", BCP 14, RFC 2119, 2218 DOI 10.17487/RFC2119, March 1997, 2219 . 2221 [RFC2315] Kaliski, B., "PKCS #7: Cryptographic Message Syntax 2222 Version 1.5", RFC 2315, DOI 10.17487/RFC2315, March 1998, 2223 . 2225 [RFC3394] Schaad, J. and R. Housley, "Advanced Encryption Standard 2226 (AES) Key Wrap Algorithm", RFC 3394, DOI 10.17487/RFC3394, 2227 September 2002, . 2229 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 2230 Thayer, "OpenPGP Message Format", RFC 4880, 2231 DOI 10.17487/RFC4880, November 2007, 2232 . 2234 [RFC4949] Shirey, R., "Internet Security Glossary, Version 2", 2235 FYI 36, RFC 4949, DOI 10.17487/RFC4949, August 2007, 2236 . 2238 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2239 Key Derivation Function (HKDF)", RFC 5869, 2240 DOI 10.17487/RFC5869, May 2010, 2241 . 2243 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2244 Specifications and Registration Procedures", BCP 13, 2245 RFC 6838, DOI 10.17487/RFC6838, January 2013, 2246 . 2248 [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data 2249 Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2250 2014, . 2252 [RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web 2253 Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2254 2015, . 2256 [RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", 2257 RFC 7516, DOI 10.17487/RFC7516, May 2015, 2258 . 2260 [RFC7517] Jones, M., "JSON Web Key (JWK)", RFC 7517, 2261 DOI 10.17487/RFC7517, May 2015, 2262 . 2264 [RFC7518] Jones, M., "JSON Web Algorithms (JWA)", RFC 7518, 2265 DOI 10.17487/RFC7518, May 2015, 2266 . 2268 18. Informative References 2270 [BLOCKCHAIN] 2271 Chain.com, "Blockchain Specification". 2273 [Davis2001] 2274 Davis, D., "Defective Sign & Encrypt in S/MIME, PKCS#7, 2275 MOSS, PEM, PGP, and XML", May 2001. 2277 [RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, 2278 RFC 5652, DOI 10.17487/RFC5652, September 2009, 2279 . 2281 [ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format 2282 Specification", October 2014.