idnits 2.17.1 draft-hallambaker-mesh-dare-07.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Authors' Addresses Section. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (9 March 2020) is 1481 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- ** Obsolete normative reference: RFC 7159 (Obsoleted by RFC 8259) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. M. Hallam-Baker 3 Internet-Draft ThresholdSecrets.com 4 Intended status: Informational 9 March 2020 5 Expires: 10 September 2020 7 Mathematical Mesh 3.0 Part III : Data At Rest Encryption (DARE) 8 draft-hallambaker-mesh-dare-07 10 Abstract 12 This document describes the Data At Rest Encryption (DARE) Envelope 13 and Container syntax. 15 The DARE Envelope syntax is used to digitally sign, digest, 16 authenticate, or encrypt arbitrary content data. 18 The DARE Container syntax describes an append-only sequence of 19 entries, each containing a DARE Envelope. DARE Containers may 20 support cryptographic integrity verification of the entire data 21 container content by means of a Merkle tree. 23 [Note to Readers] 25 Discussion of this draft takes place on the MATHMESH mailing list 26 (mathmesh@ietf.org), which is archived at 27 https://mailarchive.ietf.org/arch/search/?email_list=mathmesh. 29 This document is also available online at 30 http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at https://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on 10 September 2020. 49 Copyright Notice 51 Copyright (c) 2020 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 56 license-info) in effect on the date of publication of this document. 57 Please review these documents carefully, as they describe your rights 58 and restrictions with respect to this document. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 63 1.1. Encryption and Integrity . . . . . . . . . . . . . . . . 5 64 1.1.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 5 65 1.1.2. Data Erasure . . . . . . . . . . . . . . . . . . . . 6 66 1.2. Signature . . . . . . . . . . . . . . . . . . . . . . . . 6 67 1.2.1. Signing Individual Plaintext Envelopes . . . . . . . 7 68 1.2.2. Signing Individual Encrypted Envelopes . . . . . . . 7 69 1.2.3. Signing sequences of envelopes . . . . . . . . . . . 7 70 1.3. Container . . . . . . . . . . . . . . . . . . . . . . . . 8 71 1.3.1. Container Format . . . . . . . . . . . . . . . . . . 8 72 1.3.2. Write . . . . . . . . . . . . . . . . . . . . . . . . 9 73 1.3.3. Encryption and Authentication . . . . . . . . . . . . 9 74 1.3.4. Integrity and Signature . . . . . . . . . . . . . . . 10 75 1.3.5. Redaction . . . . . . . . . . . . . . . . . . . . . . 10 76 1.3.6. Alternative approaches . . . . . . . . . . . . . . . 11 77 1.3.7. Efficiency . . . . . . . . . . . . . . . . . . . . . 11 78 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 12 79 2.1. Related Specifications . . . . . . . . . . . . . . . . . 12 80 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 12 81 2.3. Defined terms . . . . . . . . . . . . . . . . . . . . . . 13 82 3. DARE Envelope Architecture . . . . . . . . . . . . . . . . . 14 83 3.1. Processing Considerations . . . . . . . . . . . . . . . . 15 84 3.2. Content Metadata and Annotations . . . . . . . . . . . . 15 85 3.3. Encoded Data Sequence . . . . . . . . . . . . . . . . . . 16 86 3.4. Encryption and Integrity . . . . . . . . . . . . . . . . 17 87 3.4.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 17 88 3.4.2. Key Identifiers . . . . . . . . . . . . . . . . . . . 18 89 3.4.3. Salt Derivation . . . . . . . . . . . . . . . . . . . 19 90 3.4.4. Key Derivation . . . . . . . . . . . . . . . . . . . 19 91 3.5. Signature . . . . . . . . . . . . . . . . . . . . . . . . 20 92 3.6. Algorithms . . . . . . . . . . . . . . . . . . . . . . . 20 93 3.6.1. Field: kwd . . . . . . . . . . . . . . . . . . . . . 20 94 4. DARE Container Architecture . . . . . . . . . . . . . . . . . 20 95 4.1. Container Navigation . . . . . . . . . . . . . . . . . . 21 96 4.1.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . 22 97 4.1.2. Position Index . . . . . . . . . . . . . . . . . . . 22 98 4.1.3. Metadata Index . . . . . . . . . . . . . . . . . . . 22 99 4.2. Integrity Mechanisms . . . . . . . . . . . . . . . . . . 22 100 4.2.1. Digest Chain calculation . . . . . . . . . . . . . . 23 101 4.2.2. Binary Merkle tree calculation . . . . . . . . . . . 23 102 4.2.3. Signature . . . . . . . . . . . . . . . . . . . . . . 23 103 5. DARE Schema . . . . . . . . . . . . . . . . . . . . . . . . . 24 104 5.1. Message Classes . . . . . . . . . . . . . . . . . . . . . 24 105 5.1.1. Structure: DareEnvelopeSequence . . . . . . . . . . . 24 106 5.2. Header and Trailer Classes . . . . . . . . . . . . . . . 25 107 5.2.1. Structure: DareTrailer . . . . . . . . . . . . . . . 25 108 5.2.2. Structure: DareHeader . . . . . . . . . . . . . . . . 25 109 5.2.3. Structure: ContentMeta . . . . . . . . . . . . . . . 26 110 5.3. Cryptographic Data . . . . . . . . . . . . . . . . . . . 27 111 5.3.1. Structure: DareSigner . . . . . . . . . . . . . . . . 27 112 5.3.2. Structure: X509Certificate . . . . . . . . . . . . . 28 113 5.3.3. Structure: DareSignature . . . . . . . . . . . . . . 28 114 5.3.4. Structure: DareRecipient . . . . . . . . . . . . . . 28 115 6. DARE Container Schema . . . . . . . . . . . . . . . . . . . . 28 116 6.1. Container Headers . . . . . . . . . . . . . . . . . . . . 29 117 6.1.1. Structure: ContainerInfo . . . . . . . . . . . . . . 29 118 6.2. Index Structures . . . . . . . . . . . . . . . . . . . . 29 119 6.2.1. Structure: ContainerIndex . . . . . . . . . . . . . . 29 120 6.2.2. Structure: IndexPosition . . . . . . . . . . . . . . 30 121 6.2.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 30 122 6.2.4. Structure: IndexMeta . . . . . . . . . . . . . . . . 30 123 7. Dare Container Applications . . . . . . . . . . . . . . . . . 31 124 7.1. Catalog . . . . . . . . . . . . . . . . . . . . . . . . . 31 125 7.2. Spool . . . . . . . . . . . . . . . . . . . . . . . . . . 31 126 7.3. Archive . . . . . . . . . . . . . . . . . . . . . . . . . 32 127 8. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 32 128 8.1. Terminal integrity check . . . . . . . . . . . . . . . . 32 129 8.2. Terminal index record . . . . . . . . . . . . . . . . . . 33 130 8.3. Deferred indexing . . . . . . . . . . . . . . . . . . . . 33 131 9. Security Considerations . . . . . . . . . . . . . . . . . . . 33 132 9.1. Encryption/Signature nesting . . . . . . . . . . . . . . 34 133 9.2. Side channel . . . . . . . . . . . . . . . . . . . . . . 34 134 9.3. Salt reuse . . . . . . . . . . . . . . . . . . . . . . . 34 135 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 136 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34 137 12. Appendix A: DARE Envelope Examples and Test Vectors . . . . . 34 138 13. Test Examples . . . . . . . . . . . . . . . . . . . . . . . . 34 139 13.1. Plaintext Message . . . . . . . . . . . . . . . . . . . 35 140 13.2. Plaintext Message with EDS . . . . . . . . . . . . . . . 35 141 13.3. Encrypted Message . . . . . . . . . . . . . . . . . . . 35 142 13.4. Signed Message . . . . . . . . . . . . . . . . . . . . . 37 143 13.5. Signed and Encrypted Message . . . . . . . . . . . . . . 37 144 14. Appendix B: DARE Container Examples and Test Vectors . . . . 38 145 14.1. Simple container . . . . . . . . . . . . . . . . . . . . 38 146 14.2. Payload and chain digests . . . . . . . . . . . . . . . 39 147 14.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . 41 148 14.4. Signed container . . . . . . . . . . . . . . . . . . . . 43 149 14.5. Encrypted container . . . . . . . . . . . . . . . . . . 44 150 15. Appendix C: Previous Frame Function . . . . . . . . . . . . . 47 151 16. Appendix D: Outstanding Issues . . . . . . . . . . . . . . . 47 152 17. Normative References . . . . . . . . . . . . . . . . . . . . 48 153 18. Informative References . . . . . . . . . . . . . . . . . . . 49 155 1. Introduction 157 This document describes the Data At Rest Encryption (DARE) Envelope 158 and Container Syntax. The DARE Envelope syntax is used to digitally 159 sign, digest, authenticate, or encrypt arbitrary message content. 160 The DARE Container syntax describes an append-only sequence of data 161 frames, each containing a DARE Envelope that supports efficient 162 incremental signature and encryption. 164 The DARE Envelope Syntax is based on a subset of the JSON Web 165 Signature [RFC7515] and JSON Web Encryption [RFC7516] standards and 166 shares many fields and semantics. The processing model and data 167 structures have been streamlined to remove alternative means of 168 specifying the same content and to enable multiple data sequences to 169 be signed and encrypted under a single master encryption key without 170 compromise to security. 172 A DARE Envelope consists of a _Header_, _Payload_ and an optional 173 _Trailer_. To enable single pass encoding and decoding, the Header 174 contains all the information required to perform cryptographic 175 processing of the Payload and authentication data (digest, MAC, 176 signature values) MAY be deferred to the Trailer section. 178 A DARE Container is an append-only log format consisting of a 179 sequence of frames. Cryptographic enhancements (signature, 180 encryption) may be applied to individual frames or to sets of frames. 181 Thus, a single key exchange may be used to provide a master key to 182 encrypt multiple frames and a single signature may be used to 183 authenticate all the frames in the container up to and including the 184 frame in which the signature is presented. 186 The DARE Envelope syntax may be used either as a standalone 187 cryptographic message syntax or as a means of presenting a single 188 DARE Container frame together with the complete cryptographic context 189 required to verify the contents and decrypt them. 191 1.1. Encryption and Integrity 193 A key innovation in the DARE Envelope Syntax is the separation of key 194 exchange and data encryption operations so that a Master Key (MK) 195 established in a single exchange to be applied to multiple data 196 sequences. This means that a single public key operation MAY be used 197 to encrypt and/or authenticate multiple parts of the same DARE 198 Envelope or multiple frames in a DARE Container. 200 To avoid reuse of the key and to avoid the need to communicate 201 separate IVs, each octet sequence is encrypted under a different 202 encryption key (and IV if required) derived from the Master Key by 203 means of a salt that is unique for each octet sequence that is 204 encrypted. The same approach is used to generate keys for 205 calculating a MAC over the octet sequence if required. This approach 206 allows encryption and integrity protections to be applied to the 207 envelope payload, to header or trailer fields or to application 208 defined Enhanced Data Sequences in the header or trailer. 210 1.1.1. Key Exchange 212 Traditional cryptographic containers describe the application of a 213 single key exchange to encryption of a single octet sequence. 214 Examples include PCKS#7/CMS [RFC2315], OpenPGP [RFC4880] and JSON Web 215 Encryption [RFC7516]. 217 To encrypt data using RSA, the encoder first generates a random 218 encryption key and initialization vector (IV). The encryption key is 219 encrypted under the public key of each recipient to create a per- 220 recipient decryption entry. The encryption key, plaintext and IV are 221 used to generate the ciphertext (figure 1). 223 (Artwork only available as svg: No external link available, see 224 draft-hallambaker-mesh-dare-07.html for artwork.) 226 Figure 1 228 This approach is adequate for the task of encrypting a single octet 229 stream. It is less than satisfactory when encrypting multiple octet 230 streams or very long streams for which a rekeying operation is 231 desirable. 233 In the DARE approach, key exchange and key derivation are separate 234 operations and keys MAY be derived for encryption or integrity 235 purposes or both. A single key exchange MAY be used to derive keys 236 to apply encryption and integrity enhancements to multiple data 237 sequences. 239 The DARE key exchange begins with the same key exchange used to 240 produce the CEK in JWE but instead of using the CEK to encipher data 241 directly, it is used as one of the inputs to a Key Derivation 242 Function (KDF) that is used to derive parameters for each block of 243 data to be encrypted. To avoid the need to introduce additional 244 terminology, the term 'CEK' is still used to describe the output of 245 the key agreement algorithm (including key unwrapping if required) 246 but it is more appropriately described as a Master Key (figure 2). 248 (Artwork only available as svg: No external link available, see 249 draft-hallambaker-mesh-dare-07.html for artwork.) 251 Figure 2 253 A Master Key may be used to encrypt any number of data items. Each 254 data item is encrypted under a different encryption key and IV (if 255 required). This data is derived from the Master Key using the HKDF 256 function [RFC5869] using a different salt for each data item and 257 separate info tags for each cryptographic function (figure 3). 259 (Artwork only available as svg: No external link available, see 260 draft-hallambaker-mesh-dare-07.html for artwork.) 262 Figure 3 264 This approach to encryption offers considerably greater flexibility 265 allowing the same format for data item encryption to be applied at 266 the transport, message or field level. 268 1.1.2. Data Erasure 270 Each encrypted DARE Envelope specifies a unique Master Salt value of 271 at least 128 bits which is used to derive the salt values used to 272 derive cryptographic keys for the envelope payload and annotations. 274 Erasure of the Master Salt value MAY be used to effectively render 275 the envelope payload and annotations undecipherable without altering 276 the envelope payload data. The work factor for decryption will be 277 O(2^128) even if the decryption key is compromised. 279 1.2. Signature 281 As with encryption, DARE Envelope signatures MAY be applied to an 282 individual envelope or a sequence of envelope. 284 1.2.1. Signing Individual Plaintext Envelopes 286 When an individual plaintext envelope is signed, the digest value 287 used to create the signature is calculated over the binary value of 288 the payload data. That is, the value of the payload before the 289 encoding (Base-64, JSON-B) is applied. 291 1.2.2. Signing Individual Encrypted Envelopes 293 When an individual plaintext envelope is signed, the digest value 294 used to create the signature is calculated over the binary value of 295 the payload data. That is, the value of the payload after encryption 296 but before the encoding (Base-64, JSON-B) is applied. 298 Use of signing and encryption in combination presents the risk of 299 subtle attacks depending on the order in which signing and encryption 300 take place [Davis2001]. 302 Na?ve approaches in which an envelope is encrypted and then signed 303 present the possibility of a surreptitious forwarding attack. For 304 example, Alice signs an envelope and sends it to Mallet who then 305 strips off Alice's signature and sends the envelope to Bob. 307 Na?ve approaches in which an envelope is signed and then encrypted 308 present the possibility of an attacker claiming authorship of a 309 ciphertext. For example, Alice encrypts a ciphertext for Bob and 310 then signs it. Mallet then intercepts the envelope and sends it to 311 Bob. 313 While neither attack is a concern in all applications, both attacks 314 pose potential hazards for the unwary and require close inspection of 315 application protocol design to avoid exploitation. 317 To prevent these attacks, each signature on an envelope that is 318 signed and encrypted MUST include a witness value that is calculated 319 by applying a MAC function to the signature value as described in 320 section XXX. 322 1.2.3. Signing sequences of envelopes 324 To sign multiple envelopes with a single signature, we first 325 construct a Merkle tree of the envelope payload digest values and 326 then sign the root of the Merkle tree. 328 [This is not yet implemented but will be soon.] 330 1.3. Container 332 DARE Container is a message and file syntax that allows a sequence of 333 data frames to be represented with cryptographic integrity, 334 signature, and encryption enhancements to be constructed in an append 335 only format. 337 The format is designed to meet the requirements of a wide range of 338 use cases including: 340 * Recording transactions in persistent storage. 342 * Synchronizing transaction logs between hosts. 344 * File archive. 346 * Message spool. 348 * Signing and encrypting single data items. 350 * Incremental encryption and authentication of server logs. 352 1.3.1. Container Format 354 A Container consists of a sequence of variable length Frames. Each 355 frame consists of a forward length indicator, the framed data and a 356 reverse length indicator. The reverse length indicator is written 357 out backwards allowing the length and thus the frame to be read in 358 the reverse direction: 360 (Artwork only available as svg: No external link available, see 361 draft-hallambaker-mesh-dare-07.html for artwork.) 363 Figure 4 365 Each frame contains a single DARE Envelope consisting of a Header, 366 Payload and Trailer (if required). The first frame in a container 367 describes the container format options and defaults. These include 368 the range of encoding options for frame metadata supported and the 369 container profiles to which the container conforms. 371 All internal data formats support use of pointers of up to 64 bits 372 allowing containers of up to 18 exabytes to be written. 374 Five container types are currently specified: 376 Simple The container does not provide any index or content integrity 377 checks. 379 Tree Frame headers contain entries that specify the start position 380 of previous frames at the apex of the immediately enclosing binary 381 tree. This enables efficient random access to any frame in the 382 file. 384 Digest Each frame trailer contains a "PayloadDigest" field. 385 Modification of the payload will cause verification of the 386 "PayloadDigest" value to fail on that frame. 388 Chain Each frame trailer contains "PayloadDigest" and "ChainDigest" 389 fields allowing modifications to the payload data to be detected. 390 Modification of the payload will cause verification of the 391 "PayloadDigest" value to fail on that frame and verification of 392 the "ChainDigest" value to fail on all subsequent frames. 394 Merkle Tree Frame headers contain entries that specify the start 395 position of previous frames at the apex of the immediately 396 enclosing binary tree. Frame Trailers contain TreeDigestPartial 397 and TreeDigestFinal entries forming a Merkle digest tree. 399 1.3.2. Write 401 In normal circumstances, Containers are written as an append only 402 log. As with Envelopes, integrity information (payload digest, 403 signatures) is written to the entry trailer. Thus, large payloads 404 may be written without the need to buffer the payload data _provided 405 that_ the content length is known in advance. 407 Should exceptional circumstances require, Container entries MAY be 408 erased by overwriting the Payload and/or parts of the Header content 409 without compromising the ability to verify other entries in the 410 container. If the entry Payload is encrypted, it is sufficient to 411 erase the container salt value to render the container entry 412 effectively inaccessible (though recovery might still be possible if 413 the original salt value can be recovered from the storage media. 415 1.3.3. Encryption and Authentication 417 Frame payloads and associated attributes MAY be encrypted and/or 418 authenticated in the same manner as Envelopes. 420 _Incremental encryption_ is supported allowing encryption parameters 421 from a single public key exchange operation to be applied to encrypt 422 multiple frames. The public key exchange information is specified in 423 the first encrypted frame and subsequent frames encrypted under those 424 parameters specify the location at which the key exchange information 425 is to be found by means of the ExchangePosition field which MUST 426 specify a location that is earlier in the file. 428 To avoid cryptographic vulnerabilities resulting from key re-use, the 429 DARE key exchange requires that each encrypted sequence use an 430 encryption key and initialization vector derived from the master key 431 established in the public key exchange by means of a unique salt. 433 Each Envelope and by extension, each Container frame MUST specify a 434 unique salt value of at least 128 bits. Since the encryption key is 435 derived from the salt value by means of a Key Derivation Function, 436 erasure of the salt MAY be used as a means of rendering the payload 437 plaintext value inaccessible without changing the payload value. 439 1.3.4. Integrity and Signature 441 Signatures MAY be applied to a payload digest, the final digest in a 442 chain or tree. The chain and tree digest modes allow a single 443 signature to be used to authenticate all frame payloads in a 444 container. 446 The tree signature mode is particularly suited to applications such 447 as file archives as it allows files to be verified individually 448 without requiring the signer to sign each individually. Furthermore, 449 in applications such as code signing, it allows a single signature to 450 be used to verify both the integrity of the code and its membership 451 of the distribution. 453 As with DARE Envelope, the signature mechanism does not specify the 454 interpretation of the signature semantics. The presence of a 455 signature demonstrates that the holder of the private key applied it 456 to the specified digest value but not their motive for doing so. 457 Describing such semantics is beyond the scope of this document and is 458 deferred to future work. 460 1.3.5. Redaction 462 The chief disadvantage of using an append-only format is that 463 containers only increase in size. In many applications, much of the 464 data in the container becomes redundant or obsolete and a process 465 analogous to garbage collection is required. This process is called 466 _redaction_. 468 The simplest method of redaction is to create a new container and 469 sequentially copy each entry from the old container to the new, 470 discarding redundant frames and obsolete header information. 472 For example, partial index records may be consolidated into a single 473 index record placed in the last frame of the container. Unnecessary 474 signature and integrity data may be discarded and so on. 476 While redaction could in principle be effected by moving data in- 477 place in the existing container, supporting this approach in a robust 478 fashion is considerably more complex as it requires backward 479 references in subsequent frames to be overridden as each frame is 480 moved. 482 1.3.6. Alternative approaches 484 Many file proprietary formats are in use that support some or all of 485 these capabilities but only a handful have public, let alone open, 486 standards. DARE Container is designed to provide a superset of the 487 capabilities of existing message and file syntaxes, including: 489 * Cryptographic Message Syntax [RFC5652] defines a syntax used to 490 digitally sign, digest, authenticate, or encrypt arbitrary message 491 content. 493 * The.ZIP File Format specification [ZIPFILE] developed by Phil 494 Katz. 496 * The BitCoin Block chain [BLOCKCHAIN]. 498 * JSON Web Encryption and JSON Web Signature 500 Attempting to make use of these specifications in a layered fashion 501 would require at least three separate encoders and introduce 502 unnecessary complexity. Furthermore, there is considerable overlap 503 between the specifications providing multiple means of achieving the 504 same ends, all of which must be supported if decoders are to work 505 reliably. 507 1.3.7. Efficiency 509 Every data format represents a compromise between different concerns, 510 in particular: 512 Compactness The space required to record data in the encoding. 514 Memory Overhead The additional volatile storage (RAM) required to 515 maintain indexes etc. to support efficient retrieval operations. 517 Number of Operations The number of operations required to retrieve 518 data from or append data to an existing encoded sequence. 520 Number of Disk Seek Operations Optimizing the response time of 521 magnetic storage media to random access read requests has 522 traditionally been one of the central concerns of database design. 524 The DARE Container format is designed to the assumption that this 525 will cease to be a concern as solid state media replaces magnetic. 527 While the cost of storage of all types has declined rapidly over the 528 past decades, so has the amount of data to be stored. DARE Container 529 represents a pragmatic balance of these considerations for current 530 technology. In particular, since payload volumes are likely to be 531 very large, memory and operational efficiency are considered higher 532 priorities than compactness. 534 2. Definitions 536 2.1. Related Specifications 538 The DARE Envelope and Container formats are based on the following 539 existing standards and specifications. 541 Object serialization The JSON-B [draft-hallambaker-jsonbcd] encoding 542 is used for object serialization. This encoding is an extension 543 of the JavaScript Object Notation (JSON) [RFC7159]. 545 Message syntax The cryptographic processing model is based on JSON 546 Web Signature (JWS) [RFC7515], JSON Web Encryption (JWE) [RFC7516] 547 and JSON Web Key (JWK) [RFC7517]. 549 Cryptographic primitives. The HMAC-based Extract-and-Expand Key 550 Derivation Function [RFC5869] and Advanced Encryption Standard 551 (AES) Key Wrap with Padding Algorithm [RFC3394] are used. 553 The Uniform Data Fingerprint method of presenting data digests is 554 used for key identifiers and other purposes 555 [draft-hallambaker-mesh-udf]. 557 Cryptographic algorithms The cryptographic algorithms and 558 identifiers described in JSON Web Algorithms (JWA) [RFC7518] are 559 used together with additional algorithms as defined in the JSON 560 Object Signing and Encryption IANA registry [IANAJOSE]. 562 2.2. Requirements Language 564 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 565 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 566 document are to be interpreted as described in [RFC2119]. 568 2.3. Defined terms 570 The terms "Authentication Tag", "Content Encryption Key", "Key 571 Management Mode", "Key Encryption", "Direct Key Agreement", "Key 572 Agreement with Key Wrapping" and "Direct Encryption" are defined in 573 the JWE specification [RFC7516]. 575 The terms "Authentication", "Ciphertext", "Digital Signature", 576 "Encryption", "Initialization Vector (IV)", "Message Authentication 577 Code (MAC)", "Plaintext" and "Salt" are defined by the Internet 578 Security Glossary, Version 2 [RFC4949]. 580 Annotated Envelope A DARE Envelope that contains an "Annotations" 581 field with at least one entry. 583 Authentication Data A Message Authentication Code or authentication 584 tag. 586 Complete Envelope A DARE envelope that contains the key exchange 587 information necessary for the intended recipient(s) to decrypt it. 589 Detached Envelope A DARE envelope that does not contain the key 590 exchange information necessary for the intended recipient(s) to 591 decrypt it. 593 Encryption Context The master key, encryption algorithms and 594 associated parameters used to generate a set of one or more 595 enhanced data sequences. 597 Encoded data sequence (EDS) A sequence consisting of a salt, content 598 data and authentication data (if required by the encryption 599 context). 601 Enhancement Applying a cryptographic operation to a data sequence. 602 This includes encryption, authentication and both at the same 603 time. 605 Generator The party that generates a DARE envelope. 607 Group Encryption Key A key used to encrypt data to be read by a 608 group of users. This is typically achieved by means of some form 609 of proxy re-encryption or distributed key generation. 611 Group Encryption Key Identifier A key identifier for a group 612 encryption key. 614 Master Key (MK) The master secret from which keys are derived for 615 authenticating enhanced data sequences. 617 Recipient Any party that receives and processes at least some part 618 of a DARE envelope. 620 Related Envelope A set of DARE envelopes that share the same key 621 exchange information and hence the same Master Key. 623 Uniform Data Fingerprint (UDF) The means of presenting the result of 624 a cryptographic digest function over a data sequence and content 625 type identifier specified in the Uniform Data Fingerprint 626 specification [draft-hallambaker-mesh-udf] 628 3. DARE Envelope Architecture 630 A DARE Envelope is a sequence of three parts: 632 Header A JSON object containing information a reader requires to 633 begin processing the envelope. 635 Payload An array of octets. 637 Trailer A JSON object containing information calculated from the 638 envelope payload. 640 For example, the following sequence is a JSON encoded Envelope with 641 an empty header, a payload of zero length and an empty trailer: 643 [ {}, "", {} ] 645 DARE Envelopes MAY be encoded using JSON serialization or a binary 646 serialization for greater efficiency. 648 JSON [RFC7159] Offers compatibility with applications and libraries 649 that support JSON. Payload data is encoded using Base64 incurring 650 a 33% overhead. 652 JSON-B [draft-hallambaker-jsonbcd] A superset of JSON encoding that 653 permits binary data to be encoded as a sequence of length-data 654 segments. This avoids the Base64 overhead incurred by JSON 655 encoding. Since JSON-B is a superset of JSON encoding, an 656 application can use a single decoder for either format. 658 JSON-C [draft-hallambaker-jsonbcd] A superset of JSON-C which 659 provides additional efficiency by allowing field tags and other 660 repeated string data to be encoded by reference to a dictionary. 661 Since JSON-C is a superset of JSON and JSON-B encodings, an 662 application can use a single decoder for all three formats. 664 DARE Envelope processors MUST support JSON serialization and SHOULD 665 support JSON-B serialization. 667 3.1. Processing Considerations 669 The DARE Envelope Syntax supports single pass encoding and decoding 670 without buffering of data. All the information required to begin 671 processing a DARE envelope (key agreement information, digest 672 algorithms), is provided in the envelope header. All the information 673 that is derived from envelope processing (authentication codes, 674 digest values, signatures) is presented in the envelope trailer. 676 The choice of envelope encoding does not affect the semantics of 677 envelope processing. A DARE Envelope MAY be reserialized under the 678 same serialization or converted from any of the specified 679 serialization to any other serialization without changing the 680 semantics or integrity properties of the envelope. 682 3.2. Content Metadata and Annotations 684 A header MAY contain header fields describing the payload content. 685 These include: 687 ContentType Specifies the IANA Media Type [RFC6838]. 689 Annotations A list of Encoded Data Sequences that provide 690 application specific annotations to the envelope. 692 For example, consider the following mail message: 694 From: Alice@example.com 695 To: bob@example.com 696 Subject: TOP-SECRET Product Launch Today! 698 The CEO told me the product launch is today. Tell no-one! 700 Existing encryption approaches require that header fields such as the 701 subject line be encrypted with the body of the message or not 702 encrypted at all. Neither approach is satisfactory. In this 703 example, the subject line gives away important information that the 704 sender probably assumed would be encrypted. But if the subject line 705 is encrypted together with the message body, a mail client must 706 retrieve at least part of the message body to provide a 'folder' 707 view. 709 The plaintext form of the equivalent DARE Message encoding is: 711 [{ 712 "Annotations":["iAEBiBdGcm9tOiBBbGljZUBleGFtcGxlLmNvbYgA", 713 "iAECiBNUbzogYm9iQGV4YW1wbGUuY29tiAA", 714 "iAEDiClTdWJqZWN0OiBUT1AtU0VDUkVUIFByb2R1Y3QgTGF1bmNoIFRvZGF5 715 IYgA" 716 ], 717 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS1t 718 YWlsIn0"}, 719 "VGhlIENFTyB0b2xkIG1lIHRoZSBwcm9kdWN0IGxhdW5jaCBpcyB0b2RheS4gVGVs 720 bCBuby1vbmUh" 721 ] 723 This contains the same information as before but the mail message 724 headers are now presented as a list of Encoded Data Sequences. 726 3.3. Encoded Data Sequence 728 An encoded data sequence (EDS) is a sequence of octets that encodes a 729 data sequence according to cryptographic enhancements specified in 730 the context in which it is presented. An EDS MAY be encrypted and 731 MAY be authenticated by means of a MAC. The keys and other 732 cryptographic parameters used to apply these enhancements are derived 733 from the cryptographic context and a Salt prefix specified in the EDS 734 itself. 736 An EDS sequence contains exactly three binary fields encoded in 737 JSON-B serialization as follows: 739 Salt Prefix A sequence of octets used to derive the encryption key, 740 Initialization Vector and MAC key as required. 742 Body The plaintext or encrypted content. 744 Authentication Tag The authentication code value in the case that 745 the cryptographic context specifies use of authenticated 746 encryption or a MAC, otherwise is a zero-length field. 748 Requiring all three fields to be present, even in cases where they 749 are unnecessary simplifies processing at the cost of up to six 750 additional data bytes. 752 The encoding of the 'From' header of the previous example as a 753 plaintext EDS is as follows: 755 88 01 756 01 757 88 17 758 46 72 6f 6d 3a 20 41 6c 69 63 65 40 65 78 61 6d 759 70 6c 65 2e 63 6f 6d 760 88 00 762 3.4. Encryption and Integrity 764 Encryption and integrity protections MAY be applied to any DARE 765 Envelope Payload and Annotations. 767 The following is an encrypted version of the message shown earlier. 768 The payload and annotations have both increased in size as a result 769 of the block cipher padding. The header now includes Recipients and 770 Salt fields to enable the content to be decoded. 772 [{ 773 "enc":"A256CBC", 774 "Salt":"G5zzlYzvkdK3rDOKcsNSOg", 775 "Annotations":["iAEBiCARsZ2QOATGJu2w1l0xiQgO7zivSIR2A3R4PXN-7LhN 776 jw", 777 "iAECiCDL4C9Ufjb1wG4wxLlmFSxrF3NBRNijARiFcF0JhI7kNQ", 778 "iAEDiDB1k5YffBa8m16AhRupZaa98YsYHII7YCcvZR5se880tbgYpvEaYO_k 779 qIV5ab4StdY" 780 ], 781 "recipients":[{ 782 "kid":"MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 783 "epk":{ 784 "PublicKeyECDH":{ 785 "crv":"Ed25519", 786 "Public":"h5PFnSMhVuAuc7BH20btL-lhC58KvMXnOcgYmqUo6Ts"}}, 787 "wmk":"5XT2BuCqzVxpwxB4xMjHEEm7cx80brpQ7rNpUc4TjaUY5TWXVDFq 788 1w"} 789 ], 790 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS1t 791 YWlsIn0"}, 792 "A5zdjqjRY400iMFQ_shEM-IQnjVOOeBLP7nqUxu2ISv9QNen5-fPFjo-afU5KGQC 793 p3vz30gZ-iNctdzo2EVI9A" 794 ] 796 3.4.1. Key Exchange 798 The DARE key exchange is based on the JWE key exchange except that 799 encryption modes are intentionally limited and the output of the key 800 exchange is the DARE Master Key rather than the Content Encryption 801 Key. 803 A DARE Key Exchange MAY contain any number of Recipient entries, each 804 providing a means of decrypting the Master Key using a different 805 private key. 807 If the Key Exchange mechanism supports message recovery, Direct Key 808 Agreement is used, in all other cases, Key Wrapping is used. 810 This approach allows envelopes with one intended recipient to be 811 handled in the exact same fashion as envelopes with multiple 812 recipients. While this does require an additional key wrapping 813 operation, that could be avoided if an envelope has exactly one 814 intended recipient, this is offset by the reduction in code 815 complexity. 817 If the key exchange algorithm does not support message recovery (e.g. 818 Diffie Hellman and Elliptic Curve Diffie-Hellman), the HKDF Extract- 819 and-Expand Key Derivation Function is used to derive a master key 820 using the following info tag: 822 "dare-master" [64 61 72 65 2d 6d 61 73 74 65 72] Key derivation info 823 field used when deriving a master key from the output of a key 824 exchange. 826 The master key length is the maximum of the key size of the 827 encryption algorithm specified by the key exchange header, the key 828 size of the MAC algorithm specified by the key exchange header (if 829 used) and 256. 831 3.4.2. Key Identifiers 833 The JWE/JWS specifications define a kid field for use as a key 834 identifier but not how the identifier itself is constructed. All 835 DARE key identifiers are either UDF key fingerprints 836 [draft-hallambaker-mesh-udf] or Mesh/Recrypt Group Key Identifiers. 838 A UDF fingerprint is formed as the digest of an IANA content type and 839 the digested data. A UDF key fingerprint is formed with the content 840 type "application/pkix-keyinfo" and the digested data is the ASN.1 841 DER encoded PKIX certificate "keyInfo" sequence for the corresponding 842 public key. 844 A Group Key Identifier has the form @. Where 845 is a UDF key fingerprint and is the DNS 846 address of a service that provides the encryption service to support 847 decryption by group members. 849 3.4.3. Salt Derivation 851 A Master Salt is a sequence of 16 or more octets that is specified in 852 the Salt field of the header. 854 The Master Salt is used to derive salt values for the envelope 855 payload and associated encoded data sequences as follows. 857 Payload Salt = Master Salt 859 EDS Salt = Concatenate (Payload Salt Prefix, Master Salt) 861 Encoders SHOULD NOT generate salt values that exceed 1024 octets. 863 The salt value is opaque to the DARE encoding but MAY be used to 864 encode application specific semantics including: 866 * Frame number to allow reassembly of a data sequence split over a 867 sequence of envelopes which may be delivered out of order. 869 * Transmit the Master Key in the manner of a Kerberos ticket. 871 * Identify the Master Key under which the Enhanced Data Sequence was 872 generated. 874 * Enable access to the plaintext to be eliminated by erasure of the 875 encryption key. 877 For data erasure to be effective, the salt MUST be constructed so 878 that the difficulty of recovering the key is sufficiently high that 879 it is infeasible. For most purposes, a salt with 128 bits of 880 appropriately random data is sufficient. 882 3.4.4. Key Derivation 884 Encryption and/or authentication keys are derived from the Master Key 885 using a Extract-and-Expand Key Derivation Function as follows: 887 0. The Master Key and salt value are used to extract the PRK 888 (pseudorandom key) 890 1. The PRK is used to derive the algorithm keys using the 891 application specific information input for that key type. 893 The application specific information inputs are: 895 "dare-encrypt" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 896 encryption or encryption with authentication key. 898 "dare-iv" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 899 initialization vector. 901 "dare-mac" [dare-mac] To generate a Message Authentication Code key. 903 3.5. Signature 905 While encryption and integrity enhancements can be applied to any 906 part of a DARE Envelope, signatures are only applied to payload 907 digest values calculated over one or more envelope payloads. 909 The payload digest value for an envelope is calculated over the 910 binary payload data. That is, after any encryption enhancement has 911 been applied but before the envelope encoding is applied. This 912 allows envelopes to be converted from one encoding to another without 913 affecting signature verification. 915 Single Payload The signed value is the payload digest of the 916 envelope payload. 918 Multiple Payload. The signed value is the root of a Merkle Tree in 919 which the payload digest of the envelope is one of the leaves. 921 Verification of a multiple payload signature naturally requires the 922 additional digest values required to construct the Merkle Tree. 923 These are provided in the Trailer in a format that permits multiple 924 signers to reference the same tree data. 926 3.6. Algorithms 928 3.6.1. Field: kwd 930 The key wrapping and derivation algorithms. 932 Since the means of public key exchange is determined by the key 933 identifier of the recipient key, it is only necessary to specify the 934 algorithms used for key wrapping and derivation. 936 The default (and so far only) algorithm is kwd-aes-sha2-256-256. 938 Advanced Encryption Standard (AES) Key Wrap with Padding Algorithm 939 [RFC3394] is used to wrap the Master Exchange Key. AES 256 is used. 941 HMAC-based Extract-and-Expand Key Derivation Function [RFC5869] is 942 used for key derivation. SHA-2-256 is used for the hash function. 944 4. DARE Container Architecture 945 4.1. Container Navigation 947 Three means of locating frames in a container are supported: 949 Sequential Access frames sequentially starting from the start or the 950 end of the container. 952 Binary search Access any container frame by frame number in 953 O(log_(2)(n)) time by means of a binary tree constructed while the 954 container is written. 956 Index Access and container frame by frame number or by key by means 957 of an index record. 959 All DARE Containers support sequential access. Only tree and Merkle 960 tree containers support binary search access. An index frame MAY be 961 written appended to any container and provides O(1) access to any 962 frame listed in the index. 964 Two modes of compilation are considered: 966 Monolithic Frames are added to the container in a single operation, 967 e.g. file archives, 969 Incremental Additional frames are written to the container at 970 various intervals after it was originally created, e.g. server 971 logs, message spools. 973 In the monolithic mode, navigation requirements are best met by 974 writing an index frame to the end of the container when it is 975 complete. It is not necessary to construct a binary search tree 976 unless a Merkle tree integrity check is required. 978 In the incremental mode, Binary search provides an efficient means of 979 locating frames by frame number but not by key. Writing a complete 980 index to the container every _m_ write operations provides _O(m)_ 981 search access but requires O(n^(2)) storage. 983 Use of partial indexes provides a better compromise between speed and 984 efficiency. A partial index is written out every _m_ frames where 985 _m_ is a power of two. A complete index is written every time a 986 binary tree apex record is written. This approach provides for 987 O(log_(2)(n)) search with incremental compilation with approximately 988 double the overhead of the monolithic case. 990 4.1.1. Tree 992 Binary search is supported by means of the "TreePosition" parameter 993 specified in the FrameHeader. This parameter specifies the value of 994 the immediately preceding apex. 996 Calculation of the immediately preceding apex is most easily 997 described by representing the array index in binary with base of 1 998 (rather than 0). An array index that is a power of 2 (2, 4, 8, 16, 999 etc.) will be the apex of a complete tree. Every other array index 1000 has the value of the sum of a set of powers of 2 and the immediately 1001 preceding apex will be the value of the next smallest power of 2 in 1002 the sum. 1004 For example, to find the immediately preceding apex for frame 5, we 1005 add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame 1006 is 4. 1008 The values of Tree Position are shown for the first 8 frames in 1009 figure xx below: 1011 (Artwork only available as svg: No external link available, see 1012 draft-hallambaker-mesh-dare-07.html for artwork.) 1014 Figure 5 1016 An algorithm for efficiently calculating the immediately preceding 1017 apex is provided in Appendix C. 1019 4.1.2. Position Index 1021 Contains a table of frame number, position pairs pointing to prior 1022 locations in the file. 1024 4.1.3. Metadata Index 1026 Contains a list of IndexMeta entries. Each entry contains a metadata 1027 description and a list of frame indexes (not positions) of frames 1028 that match the description. 1030 4.2. Integrity Mechanisms 1032 Frame sequences in a DARE container MAY be protected against a frame 1033 insertion attack by means of a digest chain, a binary Merkle tree or 1034 both. 1036 4.2.1. Digest Chain calculation 1038 A digest chain is simple to implement but can only be verified if the 1039 full chain of values is known. Appending a frame to the chain has 1040 _O(1)_ complexity but verification has _O(n)_ complexity: 1042 (Artwork only available as svg: No external link available, see 1043 draft-hallambaker-mesh-dare-07.html for artwork.) 1045 Figure 6 1047 The value of the chain digest for the first frame (frame 0) is 1048 _H(H(null)+H(Payload_(0))), where null is a zero length octet 1049 sequence and payloadn is the sequence of payload data bytes for frame 1050 n_ 1052 The value of the chain digest for frame _n is H(H(Payload_(n-1) 1053 +H(Payloadn)), where A+B stands for concatenation of the byte 1054 sequences A and B._ 1056 4.2.2. Binary Merkle tree calculation 1058 The tree index mechanism describe earlier may be used to implement a 1059 binary Merkle tree. The value TreeDigest specifies the apex value of 1060 the tree for that node. 1062 Appending a frame to the chain has O(log_(2) (n)) complexity provided 1063 that the container format supports at least the binary tree index. 1064 Verifying a chain has O(log_(2) (n)) complexity, provided that the 1065 set of necessary digest inputs is known. 1067 To calculate the value of the tree digest for a node, we first 1068 calculate the values of all the sub trees that have their apex at 1069 that node and then calculate the digest of that value and the 1070 immediately preceding local apex. 1072 4.2.3. Signature 1074 Payload data MAY be signed using a JWS [RFC7515] as applied in the 1075 Envelope. 1077 Signatures are specified by the "Signatures" parameter in the content 1078 header. The data that the signature is calculated over is defined by 1079 the typ parameter of the Signature as follows. 1081 "Payload" The value of the "PayloadDigest" parameter 1083 "Chain" The value of the "ChainDigest" parameter 1084 "Tree" The value of the "TreeDigestFinal" parameter 1086 If the "typ" parameter is absent, the value Payload is implied. 1088 A frame MAY contain multiple signatures created with the same signing 1089 key and different typ values. 1091 The use of signatures over chain and tree digest values permit 1092 multiple frames to be validated using a single signature verification 1093 operation. 1095 5. DARE Schema 1097 A DARE Envelope consists of a Header, an Enhanced Data Sequence (EDS) 1098 and an optional trailer. This section describes the JSON data fields 1099 used to construct headers, trailers and complete messages. 1101 Wherever possible, fields from JWE, JWS and JWK have been used. In 1102 these cases, the fields have the exact same semantics. Note however 1103 that the classes in which these fields are presented have different 1104 structure and nesting. 1106 5.1. Message Classes 1108 A DARE Message contains a single DAREMessageSequence in either the 1109 JSON or Compact serialization as directed by the protocol in which it 1110 is applied. 1112 5.1.1. Structure: DareEnvelopeSequence 1114 A DARE Message containing Header, EDS and Trailer in JSON object 1115 encoding. Since a DAREMessage is almost invariably presented in JSON 1116 sequence or compact encoding, use of the DAREMessage subclass is 1117 preferred. 1119 Although a DARE Message is functionally an object, it is serialized 1120 as an ordered sequence. This ensures that the message header field 1121 will always precede the body in a serialization, this allowing 1122 processing of the header information to be performed before the 1123 entire body has been received. 1125 Header: DareHeader (Optional) The message header. May specify the 1126 key exchange data, pre-signature or signature data, cloaked 1127 headers and/or encrypted data sequences. 1129 Body: Binary (Optional) The message body 1130 Trailer: DareTrailer (Optional) The message trailer. If present, 1131 this contains the signature. 1133 5.2. Header and Trailer Classes 1135 A DARE Message sequence MUST contain a (possibly empty) DAREHeader 1136 and MAY contain a DARETrailer. 1138 5.2.1. Structure: DareTrailer 1140 A DARE Message Trailer 1142 Signatures: DareSignature [0..Many] A list of signatures. A message 1143 trailer MUST NOT contain a signatures field if the header contains 1144 a signatures field. 1146 SignedData: Binary (Optional) Contains a DAREHeader object 1148 PayloadDigest: Binary (Optional) If present, contains the digest of 1149 the Payload. 1151 ChainDigest: Binary (Optional) If present, contains the digest of 1152 the PayloadDigest values of this frame and the frame immediately 1153 preceding. 1155 TreeDigest: Binary (Optional) If present, contains the Binary Merkle 1156 Tree digest value. 1158 5.2.2. Structure: DareHeader 1160 Inherits: DareTrailer 1162 A DARE Message Header. Since any field that is present in a trailer 1163 MAY be placed in a header instead, the message header inherits from 1164 the trailer. 1166 EnvelopeID: String (Optional) Unique identifier 1168 EncryptionAlgorithm: String (Optional) The encryption algorithm as 1169 specified in JWE 1171 DigestAlgorithm: String (Optional) Digest Algorithm. If specified, 1172 tells decoder that the digest algorithm is used to construct a 1173 signature over the message payload. 1175 Salt: Binary (Optional) Salt value used to derrive cryptographic 1176 parameters for the content data. 1178 Malt: Binary (Optional) Hash of the Salt value used to derrive 1179 cryptographic parameters for the content data. This field SHOULD 1180 NOT be present if the Salt field is present. It is used to allow 1181 the salt value to be erased (thus rendering the payload content 1182 irrecoverable) without affecting the ability to calculate the 1183 payload digest value. 1185 Cloaked: Binary (Optional) If present in a header or trailer, 1186 specifies an encrypted data block containing additional header 1187 fields whose values override those specified in the message and 1188 context headers. 1190 When specified in a header, a cloaked field MAY be used to conceal 1191 metadata (content type, compression) and/or to specify an 1192 additional layer of key exchange. That applies to both the 1193 Message body and to headers specified within the cloaked header. 1195 Processing of cloaked data is described in... 1197 EDSS: Binary [0..Many] If present, the Annotations field contains a 1198 sequence of Encrypted Data Segments encrypted under the message 1199 Master Key. The interpretation of these fields is application 1200 specific. 1202 Signers: DareSigner [0..Many] A list of 'presignature' 1204 Recipients: DareRecipient [0..Many] A list of recipient key exchange 1205 information blocks. 1207 ContentMetaData: Binary (Optional) If present contains a JSON 1208 encoded ContentInfo structure which specifies plaintext content 1209 metadata and forms one of the inputs to the envelope digest value. 1211 ContainerInfo: ContainerInfo (Optional) Information that describes 1212 container information 1214 ContainerIndex: ContainerIndex (Optional) An index of records in the 1215 current container up to but not including this one. 1217 5.2.3. Structure: ContentMeta 1219 UniqueID: String (Optional) Unique object identifier 1221 Labels: String [0..Many] List of labels that are applied to the 1222 payload of the frame. 1224 KeyValues: KeyValue [0..Many] List of key/value pairs describing the 1225 payload of the frame. 1227 MessageType: String (Optional) The mesh message type 1229 ContentType: String (Optional) The content type field as specified 1230 in JWE 1232 Paths: String [0..Many] List of filename paths for the payload of 1233 the frame. 1235 Filename: String (Optional) The original filename under which the 1236 data was stored. 1238 Event: String (Optional) Operation on the header 1240 Created: DateTime (Optional) Initial creation date. 1242 Modified: DateTime (Optional) Date of last modification. 1244 First: Integer (Optional) Frame number of the first object instance 1245 value. 1247 Previous: Integer (Optional) Frame number of the immediately prior 1248 object instance value 1250 5.3. Cryptographic Data 1252 DARE Message uses the same fields as JWE and JWS but with different 1253 structure. In particular, DARE messages MAY have multiple recipients 1254 and multiple signers. 1256 5.3.1. Structure: DareSigner 1258 The signature value 1260 Dig: String (Optional) Digest algorithm hint. Specifying the digest 1261 algorithm to be applied to the message body allows the body to be 1262 processed in streaming mode. 1264 Alg: String (Optional) Key exchange algorithm 1266 KeyIdentifier: String (Optional) Key identifier of the signature 1267 key. 1269 Certificate: X509Certificate (Optional) PKIX certificate of signer. 1271 Path: X509Certificate (Optional) PKIX certificates that establish a 1272 trust path for the signer. 1274 5.3.2. Structure: X509Certificate 1276 X5u: String (Optional) URL identifying an X.509 public key 1277 certificate 1279 X5: Binary (Optional) An X.509 public key certificate 1281 5.3.3. Structure: DareSignature 1283 Inherits: DareSigner 1285 The signature value 1287 Manifest: Binary (Optional) The data description that was signed. 1289 SignatureValue: Binary (Optional) The signature value as an Enhanced 1290 Data Sequence under the message Master Key. 1292 WitnessValue: Binary (Optional) The signature witness value used on 1293 an encrypted message to demonstrate that the signature was 1294 authorized by a party with actual knowledge of the encryption key 1295 used to encrypt the message. 1297 5.3.4. Structure: DareRecipient 1299 Recipient information 1301 KeyIdentifier: String (Optional) Key identifier for the encryption 1302 key. 1304 The Key identifier MUST be either a UDF fingerprint of a key or a 1305 Group Key Identifier 1307 KeyWrapDerivation: String (Optional) The key wrapping and derivation 1308 algorithms. 1310 WrappedMasterKey: Binary (Optional) The wrapped master key. The 1311 master key is encrypted under the result of the key exchange. 1313 RecipientKeyData: String (Optional) The per-recipient key exchange 1314 data. 1316 6. DARE Container Schema 1318 TBS stuff 1320 6.1. Container Headers 1322 TBS stuff 1324 6.1.1. Structure: ContainerInfo 1326 DataEncoding: String (Optional) Specifies the data encoding for the 1327 header section of for the following frames. This value is ONLY 1328 valid in Frame 0 which MUST have a header encoded in JSON. 1329 Information that describes container information 1331 Index: Integer (Optional) The record index within the file. This 1332 MUST be unique and satisfy any additional requirements determined 1333 by the ContainerType. 1335 ContainerType: String (Optional) Specifies the container type for 1336 the following records. 1338 IsMeta: Boolean (Optional) If true, the current frame is a meta 1339 frame and does not contain a payload. 1341 Note: Meta frames MAY be present in any container. Applications 1342 MUST accept containers that contain meta frames at any position in 1343 the file. Applications MUST NOT interpret a meta frame as a data 1344 frame with an enpty payload. 1346 Default: Boolean (Optional) If set true in a persistent container, 1347 specifies that this record contains the default object for the 1348 container. 1350 TreePosition: Integer (Optional) Position of the frame containing 1351 the apex of the preceding sub-tree. 1353 IndexPosition: Integer (Optional) Specifies the position in the file 1354 at which the last index entry is to be found 1356 ExchangePosition: Integer (Optional) Specifies the position in the 1357 file at which the key exchange data is to be found 1359 6.2. Index Structures 1361 TBS stuff 1363 6.2.1. Structure: ContainerIndex 1365 A container index 1366 Full: Boolean (Optional) If true, the index is complete and contains 1367 position entries for all the frames in the file. If absent or 1368 false, the index is incremental and only contains position entries 1369 for records added since the last frame containing a 1370 ContainerIndex. 1372 Positions: IndexPosition [0..Many] List of container position 1373 entries 1375 Metas: IndexMeta [0..Many] List of container position entries 1377 6.2.2. Structure: IndexPosition 1379 Specifies the position in a file at which a specified record index is 1380 found 1382 Index: Integer (Optional) The record index within the file. 1384 Position: Integer (Optional) The record position within the file 1385 relative to the index base. 1387 6.2.3. Structure: KeyValue 1389 Specifies a key/value entry 1391 Key: String (Optional) The key 1393 Value: String (Optional) The value corresponding to the key 1395 6.2.4. Structure: IndexMeta 1397 Specifies the list of index entries at which a record with the 1398 specified metadata occurrs. 1400 Index: Integer [0..Many] List of record indicies within the file 1401 where frames matching the specified criteria are found. 1403 ContentType: String (Optional) Content type parameter 1405 Paths: String [0..Many] List of filename paths for the current 1406 frame. 1408 Labels: String [0..Many] List of labels that are applied to the 1409 current frame. 1411 7. Dare Container Applications 1413 DARE Containers are used to implement two forms of persistence store 1414 to support Mesh operations: 1416 Catalogs A set of related items which MAY be added, modified or 1417 deleted at any time. 1419 Spools A list of related items whose status MAY be changed at any 1420 time but which are immutable once added. 1422 Since DARE Containers are an append only log format, entries can only 1423 be modified or deleted by adding items to the log to change the 1424 status of previous entries. It is always possible to undo any 1425 operation on a catalog or spool unless the underlying container is 1426 purged or the individual entries modified. 1428 7.1. Catalog 1430 Catalogs contain a set of entries, each of which is distinguished by 1431 a unique identifier. 1433 Three operations are supported: 1435 Add Addition of the entry to the catalog 1437 Update Modification of the data associated with the entry excluding 1438 the identifier 1440 Delete Removal of the entry from the catalog 1442 The set of valid state transitions is defined by the Finite State 1443 machine: 1445 (Add-Update*-Delete)* 1447 Catalogs are used to represent sets of persistent objects associated 1448 with a Mesh Service Account. The user's set of contacts for example. 1449 Each contact entry may be modified many times over time but refers to 1450 the same subject for its entire lifetime. 1452 SchemaCatalog 1454 7.2. Spool 1456 Spools contain lists of entries, each of which is distinguished by a 1457 unique identifier. 1459 Four operations are supported: 1461 Post Addition of the entry to the spool 1463 Processed Marks the entry as having been processed. 1465 Unprocessed Returns the entry to the unread state. 1467 Delete Mark the entry as deleted allowing recovery of associated 1468 storage in a subsequent purge operation. 1470 The set of valid state transitions is defined by the Finite State 1471 machine: 1473 Post-(Processed| Unprocessed| Delete *) 1475 Spools are used to represent time sequence ordered entries such as 1476 lists of messages being sent or received, task queues and transaction 1477 logs. 1479 SchemaCatalog 1481 7.3. Archive 1483 A DARE Archive is a DARE Container whose entries contain files. This 1484 affords the same functionality as a traditional ZIP or tar archive 1485 but with the added cryptographic capabilities provided by the DARE 1486 format. 1488 8. Future Work 1490 The current specification describes an approach in which containers 1491 are written according to a strict append-only policy. Greater 1492 flexibility may be achieved by loosening this requirement allowing 1493 record(s) at the end of the container to be overwritten. 1495 8.1. Terminal integrity check 1497 A major concern when operating a critical service is the possibility 1498 of a hardware or power failure occurring during a write operation 1499 causing the file update to be incomplete. While most modern 1500 operating systems have effective mechanisms in place to prevent 1501 corruption of the file system itself in such circumstances, this does 1502 not provide sufficient protection at the application level. 1504 Appending a null record containing a container-specific magic number 1505 provides an effective means of detecting this circumstance that can 1506 be quickly verified. 1508 If a container specifies a terminal integrity check value in the 1509 header of frame zero, the container is considered to be in an 1510 incomplete write state if the final frame is not a null record 1511 specifying the magic number. 1513 When appending new records to such containers, the old terminal 1514 integrity check record is overwritten by the data being added and a 1515 new integrity check record appended to the end. 1517 8.2. Terminal index record 1519 A writer can maintain a complete (or partial) index of the container 1520 in its final record without additional space overhead by overwriting 1521 the prior index on each update. 1523 8.3. Deferred indexing 1525 The task of updating terminal indexes may be deferred to a time when 1526 the machine is not busy. This improves responsiveness and may avoid 1527 the need to re-index containers receiving a sequence of updates. 1529 This approach may be supported by appending new entries to the end of 1530 the container in the usual fashion and maintaining a record of 1531 containers to be updated as a separate task. 1533 When updating the index on a container that has been updated in this 1534 fashion, the writer must ensure that no data is lost even if the 1535 process is interrupted. The use of guard records and other 1536 precautions against loss of state is advised. 1538 9. Security Considerations 1540 This section describes security considerations arising from the use 1541 of DARE in general applications. 1543 Additional security considerations for use of DARE in Mesh services 1544 and applications are described in the Mesh Security Considerations 1545 guide [draft-hallambaker-mesh-security]. 1547 9.1. Encryption/Signature nesting 1549 9.2. Side channel 1551 9.3. Salt reuse 1553 10. IANA Considerations 1555 11. Acknowledgements 1557 A list of people who have contributed to the design of the Mesh is 1558 presented in [draft-hallambaker-mesh-architecture]. 1560 The name Data At Rest Encryption was proposed by Melhi Abdulhayo?lu. 1562 12. Appendix A: DARE Envelope Examples and Test Vectors 1564 13. Test Examples 1566 In the following examples, Alice's encryption private key parameters 1567 are: 1569 { 1570 "PrivateKeyECDH":{ 1571 "crv":"Ed25519", 1572 "Private":"YVDc2lAAlRRTOp49t8-cS5ncpC9nD8QwE6KCbs_Iv8I"}} 1574 Alice's signature private key parameters are: 1576 { 1577 "PrivateKeyECDH":{ 1578 "crv":"Ed25519", 1579 "Private":"P7ZGsbYzQAT47Ht6QgYlMnun7rbYhyuWZFnmmbvRMGI"}} 1581 The body of the test message is the UTF8 representation of the 1582 following string: 1584 "This is a test long enough to require multiple blocks" 1586 The EDS sequences, are the UTF8 representation of the following 1587 strings: 1589 "Subject: Message metadata should be encrypted" 1590 "2018-02-01" 1592 13.1. Plaintext Message 1594 A plaintext message without associated EDS sequences is an empty 1595 header followed by the message body: 1597 { 1598 "DareEnvelope":[{}, 1599 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1600 BibG9ja3M" 1601 ]} 1603 13.2. Plaintext Message with EDS 1605 If a plaintext message contains EDS sequences, these are also in 1606 plaintext: 1608 { 1609 "DareEnvelope":[{ 1610 "Annotations":["iAEBiC1TdWJqZWN0OiBNZXNzYWdlIG1ldGFkYXRhIHNob3 1611 VsZCBiZSBlbmNyeXB0ZWSIAA", 1612 "iAECiAoyMDE4LTAyLTAxiAA" 1613 ]}, 1614 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1615 BibG9ja3M" 1616 ]} 1618 13.3. Encrypted Message 1620 The creator generates a master session key: 1622 B8 80 46 8D 20 9A 6E 06 1D 66 51 EE C2 65 BA 4D 1623 A3 A6 E4 C5 11 24 91 EE 51 AF 67 69 85 F3 B4 F3 1625 For each recipient of the message: 1627 The creator generates an ephemeral key: 1629 { 1630 "PrivateKeyECDH":{ 1631 "crv":"Ed25519", 1632 "Private":"HcVhD-tsgNYYMWGglhSRs6GPRhCAfKnfqul4xOnWjM4"}} 1634 The key agreement value is calculated: 1636 CD D1 1C 5F 39 F5 FE 75 EE BC 4D C6 EE 41 82 0D 1637 CF 32 6E 19 06 18 B2 5D F2 2A 2C AE 0B B6 E2 29 1639 The key agreement value is used as the input to a HKDF key derivation 1640 function with the info parameter master to create the key used to 1641 wrap the master key: 1643 EE 0B 3D 1D 95 B4 8C BE 60 13 46 E9 B0 2B F8 23 1644 42 6E 7D 4A 20 4B 3C 96 DC 2C 03 FF F5 04 E2 18 1646 The wrapped master key is: 1648 77 07 5F C4 C3 02 83 FE 4A 3B 08 A2 31 46 32 1C 1649 6C AA 2C 1C 70 70 2B E7 72 08 73 55 BA 10 28 29 1650 21 38 4D 00 CA 3A 4F 7E 1652 This information is used to calculate the Recipient information shown 1653 in the example below. 1655 To encrypt a message, we first generate a unique salt value: 1657 0A 01 BA 13 4A 1F C2 4C 78 19 F0 50 60 49 32 E6 1659 The salt value and master key are used to generate the payload 1660 encryption key: 1662 E7 02 E0 B6 C1 22 86 F4 0F EB FF 8F 10 82 1F ED 1663 F3 DB 59 B9 00 62 25 BA 79 82 8E C5 B3 21 76 6C 1665 Since AES is a block cipher, we also require an initializarion 1666 vector: 1668 51 D3 87 C4 0E 3E 88 65 83 34 64 0D 5E E7 D6 6A 1670 The output sequence is the encrypted bytes: 1672 C9 79 19 E0 B7 57 4D F9 38 FB 33 D9 1F 6B B4 01 1673 B6 21 B5 16 A9 8B 1E 8B 92 29 5F 48 B6 95 1B 3D 1674 DB E1 60 24 C9 44 3E AC 0B FA 25 1E 6C E2 79 97 1675 DA 31 18 64 62 79 98 26 7E DA FB 16 8D 40 15 81 1677 Since the message is not signed, there is no need for a trailer. The 1678 completed message is: 1680 { 1681 "DareEnvelope":[{ 1682 "enc":"A256CBC", 1683 "Salt":"CgG6E0ofwkx4GfBQYEky5g", 1684 "recipients":[{ 1685 "kid":"MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 1686 "epk":{ 1687 "PublicKeyECDH":{ 1688 "crv":"Ed25519", 1689 "Public":"zN9NE_98jmR3D0HB8sbbzOs5B_Yc0basc4Z9CxDHCTc"} 1690 }, 1691 "wmk":"dwdfxMMCg_5KOwiiMUYyHGyqLBxwcCvncghzVboQKCkhOE0Ayj 1692 pPfg"} 1693 ]}, 1694 "yXkZ4LdXTfk4-zPZH2u0AbYhtRapix6LkilfSLaVGz3b4WAkyUQ-rAv6JR5s4n 1695 mX2jEYZGJ5mCZ-2vsWjUAVgQ" 1696 ]} 1698 13.4. Signed Message 1700 Signed messages specify the digest algorithm to be used in the header 1701 and the signature value in the trailer. Note that the digest 1702 algorithm is not optional since it serves as notice that a decoder 1703 should digest the payload value to enable signature verification. 1705 { 1706 "DareEnvelope":[{ 1707 "dig":"SHA2"}, 1708 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1709 BibG9ja3M", 1710 { 1711 "signatures":[{ 1712 "alg":"SHA2", 1713 "kid":"MD6B-BZ25-DIK6-76IL-S5U2-33BS-L4KQ", 1714 "signature":"sv-z5iBfBfQ2DD635M5ysM7wZ1kdiusp_3ek0L1C0tpc 1715 VOvdwIwefIY-y8gy7Wack-BjisrQr6R_t-kOlGWXDg"} 1716 ], 1717 "PayloadDigest":"raim8SV5adPbWWn8FMM4mrRAQCO9A2jZ0NZAnFXWlG0x 1718 F6sWGJbnKSdtIJMmMU_hjarlIPEoY3vy9UdVlH5KAg"} 1719 ]} 1721 13.5. Signed and Encrypted Message 1723 A signed and encrypted message is encrypted and then signed. The 1724 signer proves knowledge of the payload plaintext by providing the 1725 plaintext witness value. 1727 { 1728 "DareEnvelope":[{ 1729 "enc":"A256CBC", 1730 "dig":"SHA2", 1731 "Salt":"b1DTSOar_Gy2hEMEzH4VyA", 1732 "recipients":[{ 1733 "kid":"MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 1734 "epk":{ 1735 "PublicKeyECDH":{ 1736 "crv":"Ed25519", 1737 "Public":"NWKOzIPkWK6WRIekz0LXdGpjOu1wcqqDR0WT2fyyq2w"} 1738 }, 1739 "wmk":"ElI1Fj1xxowDiLbu0AAdb9slBZjQv9htXCsv3Dd6d1la8hhWky 1740 bOmA"} 1741 ]}, 1742 "Uypt-b72evPGc1cVSxqaspKjFTNYWIgxowQ-gn8lF8_mjoN5Thf0esf4I9cYpv 1743 5dNFDyhOym_iK5NyWFXTEBig", 1744 { 1745 "signatures":[{ 1746 "alg":"SHA2", 1747 "kid":"MD6B-BZ25-DIK6-76IL-S5U2-33BS-L4KQ", 1748 "signature":"4fEo2MdeXbp6cr97TTZXUvTpJQTu1WTAOrDFRmAzb4Ye 1749 faSrrRMvX4q1BB7q52YwqeybbW6aigpz9psRBhWbAQ", 1750 "witness":"iVib2Afu55V6CN1b3kl4HkN_4pic-wqt_tUp430tIJo"} 1751 ], 1752 "PayloadDigest":"kVieR-HvMSgpFQc4snccMW6iAfQVzkQFhlWbuB0bWA-x 1753 au2JR5e1PSCbm1vN05px2eZziNqaUw-dDxo9G1ekMA"} 1754 ]} 1756 14. Appendix B: DARE Container Examples and Test Vectors 1758 The data payloads in all the following examples are identical, only 1759 the authentication and/or encryption is different. 1761 * Frame 1..n consists of 300 bytes being the byte sequence 00, 01, 1762 02, etc. repeating after 256 bytes. 1764 For conciseness, the raw data format is omitted for examples after 1765 the first, except where the data payload has been transformed, (i.e. 1766 encrypted). 1768 14.1. Simple container 1770 the following example shows a simple container with first frame and a 1771 single data frame: 1773 f4 82 1774 f0 7c 1775 f0 00 1776 f0 00 1777 82 f4 1778 f5 01 74 1779 f0 43 1780 f1 01 2c 1781 74 01 f5 1783 Since there is no integrity check, there is no need for trailer 1784 entries. The header values are: 1786 Frame 0 1788 { 1789 "ContentMetaData": "e30", 1790 "ContainerInfo": { 1791 "DataEncoding": "JSON", 1792 "Index": 0, 1793 "ContainerType": "List"}} 1795 [Empty trailer] 1797 Frame 1 1799 { 1800 "ContentMetaData": "e30", 1801 "ContainerInfo": { 1802 "Index": 1}} 1804 [Empty trailer] 1806 14.2. Payload and chain digests 1808 The following example shows a chain container with a first frame and 1809 three data frames. The headers of these frames is the same as before 1810 but the frames now have trailers specifying the PayloadDigest and 1811 ChainDigest values: 1813 Frame 0 1814 { 1815 "ContentMetaData": "e30", 1816 "ContainerInfo": { 1817 "DataEncoding": "JSON", 1818 "Index": 0, 1819 "ContainerType": "Chain"}} 1821 [Empty trailer] 1823 Frame 1 1825 { 1826 "ContentMetaData": "e30", 1827 "ContainerInfo": { 1828 "Index": 1}} 1830 { 1831 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1832 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1833 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1834 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1836 Frame 2 1838 { 1839 "ContentMetaData": "e30", 1840 "ContainerInfo": { 1841 "Index": 2}} 1843 { 1844 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1845 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1846 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1847 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1849 Frame 3 1851 { 1852 "ContentMetaData": "e30", 1853 "ContainerInfo": { 1854 "Index": 3}} 1856 { 1857 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1858 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1859 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1860 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1862 14.3. Merkle Tree 1864 The following example shows a chain container with a first frame and 1865 six data frames. The trailers now contain the TreePosition and 1866 TreeDigest values: 1868 Frame 0 1870 { 1871 "ContentMetaData": "e30", 1872 "ContainerInfo": { 1873 "DataEncoding": "JSON", 1874 "Index": 0, 1875 "ContainerType": "Merkle"}} 1877 [Empty trailer] 1879 Frame 1 1881 { 1882 "ContentMetaData": "e30", 1883 "ContainerInfo": { 1884 "Index": 1, 1885 "TreePosition": 0}} 1887 { 1888 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1889 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1890 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1891 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1893 Frame 2 1895 { 1896 "ContentMetaData": "e30", 1897 "ContainerInfo": { 1898 "Index": 2, 1899 "TreePosition": 360}} 1901 { 1902 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1903 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1904 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 1905 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 1907 Frame 3 1908 { 1909 "ContentMetaData": "e30", 1910 "ContainerInfo": { 1911 "Index": 3, 1912 "TreePosition": 360}} 1914 { 1915 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1916 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1917 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1918 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1920 Frame 4 1922 { 1923 "ContentMetaData": "e30", 1924 "ContainerInfo": { 1925 "Index": 4, 1926 "TreePosition": 1612}} 1928 { 1929 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1930 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1931 "TreeDigest": "vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvK 1932 Sj8gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"} 1934 Frame 5 1936 { 1937 "ContentMetaData": "e30", 1938 "ContainerInfo": { 1939 "Index": 5, 1940 "TreePosition": 1612}} 1942 { 1943 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1944 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1945 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1946 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1948 Frame 6 1949 { 1950 "ContentMetaData": "e30", 1951 "ContainerInfo": { 1952 "Index": 6, 1953 "TreePosition": 2867}} 1955 { 1956 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1957 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1958 "TreeDigest": "WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9S 1959 Z0mdr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"} 1961 14.4. Signed container 1963 The following example shows a tree container with a signature in the 1964 final record. The signing key parameters are: 1966 { 1967 "PrivateKeyECDH":{ 1968 "crv":"Ed25519", 1969 "Private":"P7ZGsbYzQAT47Ht6QgYlMnun7rbYhyuWZFnmmbvRMGI"}} 1971 The container headers and trailers are: 1973 Frame 0 1975 { 1976 "ContentMetaData": "e30", 1977 "ContainerInfo": { 1978 "DataEncoding": "JSON", 1979 "Index": 0, 1980 "ContainerType": "Merkle"}} 1982 [Empty trailer] 1984 Frame 1 1986 { 1987 "ContentMetaData": "e30", 1988 "ContainerInfo": { 1989 "Index": 1, 1990 "TreePosition": 0}} 1992 { 1993 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1994 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1995 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1996 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1998 Frame 2 2000 { 2001 "ContentMetaData": "e30", 2002 "ContainerInfo": { 2003 "Index": 2, 2004 "TreePosition": 360}} 2006 { 2007 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 2008 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2009 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 2010 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 2012 14.5. Encrypted container 2014 The following example shows a container in which all the frame 2015 payloads are encrypted under the same master secret established in a 2016 key agreement specified in the first frame. 2018 Frame 0 2020 { 2021 "enc": "A256CBC", 2022 "Salt": "fcHcotFCsQli4KDqrTAi5Q", 2023 "recipients": [{ 2024 "kid": "MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 2025 "epk": { 2026 "PublicKeyECDH": { 2027 "crv": "Ed25519", 2028 "Public": "PVUAJYFQFVgLzO49lkZCL7pe5qj-6s3T7MHx92AZ8ng"}}, 2029 "wmk": "3ExPlpasWGqw-_z7EGIJM8-rWZgNwt8_bd9inlE-ZWYB5Ef7sY68sQ" 2030 }], 2031 "ContentMetaData": "e30", 2032 "ContainerInfo": { 2033 "DataEncoding": "JSON", 2034 "Index": 0, 2035 "ContainerType": "List"}} 2037 [Empty trailer] 2039 Frame 1 2040 { 2041 "enc": "A256CBC", 2042 "Salt": "lmMrQYd-BjfKfhskOHZliw", 2043 "ContentMetaData": "e30", 2044 "ContainerInfo": { 2045 "Index": 1}} 2047 [Empty trailer] 2049 Frame 2 2051 { 2052 "enc": "A256CBC", 2053 "Salt": "2zLKKKsoKkMeTB8lbPXH-g", 2054 "ContentMetaData": "e30", 2055 "ContainerInfo": { 2056 "Index": 2}} 2058 [Empty trailer] 2060 Here are the container bytes. Note that the content is now encrypted 2061 and has expanded by 25 bytes. These are the salt (16 bytes), the AES 2062 padding (4 bytes) and the JSON-B framing (5 bytes). 2064 f5 01 e5 2065 f1 01 ce 2066 f0 10 2067 f0 00 2068 e5 01 f5 2069 f5 01 b0 2070 f0 7b 2071 f1 01 30 2072 b0 01 f5 2073 f5 01 b0 2074 f0 7b 2075 f1 01 30 2076 b0 01 f5 2078 The following example shows a container in which all the frame 2079 payloads are encrypted under separate key agreements specified in the 2080 payload frames. 2082 Frame 0 2083 { 2084 "ContentMetaData": "e30", 2085 "ContainerInfo": { 2086 "DataEncoding": "JSON", 2087 "Index": 0, 2088 "ContainerType": "List"}} 2090 [Empty trailer] 2092 Frame 1 2094 { 2095 "enc": "A256CBC", 2096 "Salt": "47z0KQHD-eDI7aUcPUsovA", 2097 "recipients": [{ 2098 "kid": "MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 2099 "epk": { 2100 "PublicKeyECDH": { 2101 "crv": "Ed25519", 2102 "Public": "T8Mkrlmefhjx1OBtoirTQfX60UYX6FPjEBFnA1ow6Yo"}}, 2103 "wmk": "92XvuomsbaG-TWHLe9xrwLzvJS7Dn5bkU5s5RqRSHbfnhNoN-0Cn0Q" 2104 }], 2105 "ContentMetaData": "e30", 2106 "ContainerInfo": { 2107 "Index": 1}} 2109 [Empty trailer] 2111 Frame 2 2113 { 2114 "enc": "A256CBC", 2115 "Salt": "ALTV-PlxDTV28RkhJ4wt8g", 2116 "recipients": [{ 2117 "kid": "MAUC-ZTKR-N3D5-TZCI-K6IR-5Q5C-VKU2", 2118 "epk": { 2119 "PublicKeyECDH": { 2120 "crv": "Ed25519", 2121 "Public": "0gk_OnQKjLCYET-jv0rW963PsNGkXMfULPDZZ4TXCMk"}}, 2122 "wmk": "vVxGQX3n0lJZ5Z1q1PYkax6tiJs8mWji0DYB4_Pj3_YNSKexnKCLnQ" 2123 }], 2124 "ContentMetaData": "e30", 2125 "ContainerInfo": { 2126 "Index": 2}} 2128 [Empty trailer] 2130 15. Appendix C: Previous Frame Function 2132 public long PreviousFrame (long Frame) { 2133 long x2 = Frame + 1; 2134 long d = 1; 2136 while (x2 > 0) { 2137 if ((x2 & 1) == 1) { 2138 return x2 == 1 ? (d / 2) - 1 : Frame - d; 2139 } 2140 d = d * 2; 2141 x2 = x2 / 2; 2142 } 2143 return 0; 2144 } 2146 16. Appendix D: Outstanding Issues 2148 The following issues need to be addressed. 2150 +----------------+-----------------------------------------------+ 2151 | Issue | Description | 2152 +================+===============================================+ 2153 | X25519 | The examples currently use Edwards Curve25519 | 2154 | | for encryption. This should be Curve X25519 | 2155 +----------------+-----------------------------------------------+ 2156 | Indexing | No examples are given of indexing a container | 2157 +----------------+-----------------------------------------------+ 2158 | Archive | Should include a file archive example | 2159 +----------------+-----------------------------------------------+ 2160 | File Path | Mention the file path security issue in the | 2161 | | security considerations | 2162 +----------------+-----------------------------------------------+ 2163 | Security | Write Security considerations | 2164 | Considerations | | 2165 +----------------+-----------------------------------------------+ 2166 | AES-GCM | Switch to using AES GCM in the examples | 2167 +----------------+-----------------------------------------------+ 2168 | Witness | Complete handling of witness values. | 2169 +----------------+-----------------------------------------------+ 2170 | Schema | Complete the schema documentation | 2171 +----------------+-----------------------------------------------+ 2172 | Container Redo | Rework the container/header objects so that | 2173 | | these are separate classes and Header is an | 2174 | | entry in the Container header. | 2175 +----------------+-----------------------------------------------+ 2177 Table 1 2179 17. Normative References 2181 [draft-hallambaker-jsonbcd] 2182 Hallam-Baker, P., "Binary Encodings for JavaScript Object 2183 Notation: JSON-B, JSON-C, JSON-D", Work in Progress, 2184 Internet-Draft, draft-hallambaker-jsonbcd-15, 23 October 2185 2019, . 2188 [draft-hallambaker-mesh-architecture] 2189 Hallam-Baker, P., "Mathematical Mesh 3.0 Part I: 2190 Architecture Guide", Work in Progress, Internet-Draft, 2191 draft-hallambaker-mesh-architecture-12, 16 January 2020, 2192 . 2195 [draft-hallambaker-mesh-security] 2196 Hallam-Baker, P., "Mathematical Mesh 3.0 Part VII: 2197 Security Considerations", Work in Progress, Internet- 2198 Draft, draft-hallambaker-mesh-security-03, 16 January 2199 2020, . 2202 [draft-hallambaker-mesh-udf] 2203 Hallam-Baker, P., "Mathematical Mesh 3.0 Part II: Uniform 2204 Data Fingerprint.", Work in Progress, Internet-Draft, 2205 draft-hallambaker-mesh-udf-08, 6 January 2020, 2206 . 2209 [IANAJOSE] "[Reference Not Found!]". 2211 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2212 Requirement Levels", BCP 14, RFC 2119, 2213 DOI 10.17487/RFC2119, March 1997, 2214 . 2216 [RFC2315] Kaliski, B., "PKCS #7: Cryptographic Message Syntax 2217 Version 1.5", RFC 2315, DOI 10.17487/RFC2315, March 1998, 2218 . 2220 [RFC3394] Schaad, J. and R. Housley, "Advanced Encryption Standard 2221 (AES) Key Wrap Algorithm", RFC 3394, DOI 10.17487/RFC3394, 2222 September 2002, . 2224 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 2225 Thayer, "OpenPGP Message Format", RFC 4880, 2226 DOI 10.17487/RFC4880, November 2007, 2227 . 2229 [RFC4949] Shirey, R., "Internet Security Glossary, Version 2", 2230 FYI 36, RFC 4949, DOI 10.17487/RFC4949, August 2007, 2231 . 2233 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2234 Key Derivation Function (HKDF)", RFC 5869, 2235 DOI 10.17487/RFC5869, May 2010, 2236 . 2238 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2239 Specifications and Registration Procedures", BCP 13, 2240 RFC 6838, DOI 10.17487/RFC6838, January 2013, 2241 . 2243 [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data 2244 Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2245 2014, . 2247 [RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web 2248 Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2249 2015, . 2251 [RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", 2252 RFC 7516, DOI 10.17487/RFC7516, May 2015, 2253 . 2255 [RFC7517] Jones, M., "JSON Web Key (JWK)", RFC 7517, 2256 DOI 10.17487/RFC7517, May 2015, 2257 . 2259 [RFC7518] Jones, M., "JSON Web Algorithms (JWA)", RFC 7518, 2260 DOI 10.17487/RFC7518, May 2015, 2261 . 2263 18. Informative References 2265 [BLOCKCHAIN] 2266 Chain.com, "Blockchain Specification". 2268 [Davis2001] 2269 Davis, D., "Defective Sign & Encrypt in S/MIME, PKCS#7, 2270 MOSS, PEM, PGP, and XML", May 2001. 2272 [RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, 2273 RFC 5652, DOI 10.17487/RFC5652, September 2009, 2274 . 2276 [ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format 2277 Specification", October 2014.