idnits 2.17.1 draft-hallambaker-mesh-dare-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The abstract seems to contain references ([1]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (July 8, 2019) is 1752 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- -- Looks like a reference, but probably isn't: '1' on line 2245 -- Looks like a reference, but probably isn't: '2' on line 2247 -- Looks like a reference, but probably isn't: '3' on line 2249 -- Looks like a reference, but probably isn't: '4' on line 2251 -- Looks like a reference, but probably isn't: '5' on line 2253 -- Looks like a reference, but probably isn't: '6' on line 2255 -- Looks like a reference, but probably isn't: '7' on line 2257 ** Obsolete normative reference: RFC 7159 (Obsoleted by RFC 8259) Summary: 2 errors (**), 0 flaws (~~), 1 warning (==), 8 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. Hallam-Baker 3 Internet-Draft July 8, 2019 4 Intended status: Informational 5 Expires: January 9, 2020 7 Mathematical Mesh 3.0 Part III : Data At Rest Encryption (DARE) 8 draft-hallambaker-mesh-dare-03 10 Abstract 12 This document describes the Data At Rest Encryption (DARE) Envelope 13 and Container syntax. 15 The DARE Envelope syntax is used to digitally sign, digest, 16 authenticate, or encrypt arbitrary content data. 18 The DARE Container syntax describes an append-only sequence of 19 entries, each containing a DARE Envelope. DARE Containers may 20 support cryptographic integrity verification of the entire data 21 container content by means of a Merkle tree. 23 This document is also available online at 24 http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html [1] . 26 Status of This Memo 28 This Internet-Draft is submitted in full conformance with the 29 provisions of BCP 78 and BCP 79. 31 Internet-Drafts are working documents of the Internet Engineering 32 Task Force (IETF). Note that other groups may also distribute 33 working documents as Internet-Drafts. The list of current Internet- 34 Drafts is at https://datatracker.ietf.org/drafts/current/. 36 Internet-Drafts are draft documents valid for a maximum of six months 37 and may be updated, replaced, or obsoleted by other documents at any 38 time. It is inappropriate to use Internet-Drafts as reference 39 material or to cite them other than as "work in progress." 41 This Internet-Draft will expire on January 9, 2020. 43 Copyright Notice 45 Copyright (c) 2019 IETF Trust and the persons identified as the 46 document authors. All rights reserved. 48 This document is subject to BCP 78 and the IETF Trust's Legal 49 Provisions Relating to IETF Documents 50 (https://trustee.ietf.org/license-info) in effect on the date of 51 publication of this document. Please review these documents 52 carefully, as they describe your rights and restrictions with respect 53 to this document. Code Components extracted from this document must 54 include Simplified BSD License text as described in Section 4.e of 55 the Trust Legal Provisions and are provided without warranty as 56 described in the Simplified BSD License. 58 Table of Contents 60 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 61 1.1. Encryption and Integrity . . . . . . . . . . . . . . . . 5 62 1.1.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 5 63 1.1.2. Data Erasure . . . . . . . . . . . . . . . . . . . . 6 64 1.2. Signature . . . . . . . . . . . . . . . . . . . . . . . . 7 65 1.2.1. Signing Individual Plaintext Envelopes . . . . . . . 7 66 1.2.2. Signing Individual Encrypted Envelopes . . . . . . . 7 67 1.2.3. Signing sequences of envelopes . . . . . . . . . . . 8 68 1.3. Container . . . . . . . . . . . . . . . . . . . . . . . . 8 69 1.3.1. Container Format . . . . . . . . . . . . . . . . . . 8 70 1.3.2. Write . . . . . . . . . . . . . . . . . . . . . . . . 9 71 1.3.3. Encryption and Authentication . . . . . . . . . . . . 10 72 1.3.4. Integrity and Signature . . . . . . . . . . . . . . . 10 73 1.3.5. Redaction . . . . . . . . . . . . . . . . . . . . . . 11 74 1.3.6. Alternative approaches . . . . . . . . . . . . . . . 11 75 1.3.7. Efficiency . . . . . . . . . . . . . . . . . . . . . 12 76 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 12 77 2.1. Related Specifications . . . . . . . . . . . . . . . . . 12 78 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 13 79 2.3. Defined terms . . . . . . . . . . . . . . . . . . . . . . 13 80 3. DARE Envelope Architecture . . . . . . . . . . . . . . . . . 14 81 3.1. Processing Considerations . . . . . . . . . . . . . . . . 15 82 3.2. Content Metadata and Annotations . . . . . . . . . . . . 15 83 3.3. Encoded Data Sequence . . . . . . . . . . . . . . . . . . 16 84 3.4. Encryption and Integrity . . . . . . . . . . . . . . . . 17 85 3.4.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 18 86 3.4.2. Key Identifiers . . . . . . . . . . . . . . . . . . . 18 87 3.4.3. Salt Derivation . . . . . . . . . . . . . . . . . . . 19 88 3.4.4. Key Derivation . . . . . . . . . . . . . . . . . . . 19 89 3.5. Signature . . . . . . . . . . . . . . . . . . . . . . . . 20 90 3.6. Algorithms . . . . . . . . . . . . . . . . . . . . . . . 20 91 3.6.1. Field: kwd . . . . . . . . . . . . . . . . . . . . . 20 92 4. DARE Container Architecture . . . . . . . . . . . . . . . . . 21 93 4.1. Container Navigation . . . . . . . . . . . . . . . . . . 21 94 4.1.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . 22 95 4.1.2. Position Index . . . . . . . . . . . . . . . . . . . 22 96 4.1.3. Metadata Index . . . . . . . . . . . . . . . . . . . 22 97 4.2. Integrity Mechanisms . . . . . . . . . . . . . . . . . . 23 98 4.2.1. Digest Chain calculation . . . . . . . . . . . . . . 23 99 4.2.2. Binary Merkle tree calculation . . . . . . . . . . . 23 100 4.2.3. Signature . . . . . . . . . . . . . . . . . . . . . . 23 101 5. DARE Schema . . . . . . . . . . . . . . . . . . . . . . . . . 24 102 5.1. Message Classes . . . . . . . . . . . . . . . . . . . . . 24 103 5.1.1. Structure: DareEnvelopeSequence . . . . . . . . . . . 24 104 5.2. Header and Trailer Classes . . . . . . . . . . . . . . . 25 105 5.2.1. Structure: DareTrailer . . . . . . . . . . . . . . . 25 106 5.2.2. Structure: DareHeader . . . . . . . . . . . . . . . . 25 107 5.3. Cryptographic Data . . . . . . . . . . . . . . . . . . . 27 108 5.3.1. Structure: DareSigner . . . . . . . . . . . . . . . . 27 109 5.3.2. Structure: X509Certificate . . . . . . . . . . . . . 27 110 5.3.3. Structure: DareSignature . . . . . . . . . . . . . . 27 111 5.3.4. Structure: DareRecipient . . . . . . . . . . . . . . 28 112 6. DARE Container Schema . . . . . . . . . . . . . . . . . . . . 28 113 6.1. Container Headers . . . . . . . . . . . . . . . . . . . . 28 114 6.1.1. Structure: ContainerEntry . . . . . . . . . . . . . . 28 115 6.1.2. Structure: ContainerHeaderFirst . . . . . . . . . . . 28 116 6.1.3. Structure: ContainerHeader . . . . . . . . . . . . . 29 117 6.2. Content Metadata Structure . . . . . . . . . . . . . . . 30 118 6.2.1. Structure: ContentMeta . . . . . . . . . . . . . . . 30 119 6.3. Index Structures . . . . . . . . . . . . . . . . . . . . 30 120 6.3.1. Structure: ContainerIndex . . . . . . . . . . . . . . 30 121 6.3.2. Structure: IndexPosition . . . . . . . . . . . . . . 30 122 6.3.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 31 123 6.3.4. Structure: IndexMeta . . . . . . . . . . . . . . . . 31 124 7. Dare Container Applications . . . . . . . . . . . . . . . . . 31 125 7.1. Catalog . . . . . . . . . . . . . . . . . . . . . . . . . 31 126 7.2. Spool . . . . . . . . . . . . . . . . . . . . . . . . . . 32 127 7.3. Archive . . . . . . . . . . . . . . . . . . . . . . . . . 33 128 8. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 33 129 8.1. Terminal integrity check . . . . . . . . . . . . . . . . 33 130 8.2. Terminal index record . . . . . . . . . . . . . . . . . . 33 131 8.3. Deferred indexing . . . . . . . . . . . . . . . . . . . . 33 132 9. Security Considerations . . . . . . . . . . . . . . . . . . . 34 133 9.1. Encryption/Signature nesting . . . . . . . . . . . . . . 34 134 9.2. Side channel . . . . . . . . . . . . . . . . . . . . . . 34 135 9.3. Salt reuse . . . . . . . . . . . . . . . . . . . . . . . 34 136 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 34 137 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34 138 12. Appendix A: DARE Envelope Examples and Test Vectors . . . . . 34 139 13. Test Examples . . . . . . . . . . . . . . . . . . . . . . . . 34 140 13.1. Plaintext Message . . . . . . . . . . . . . . . . . . . 35 141 13.2. Plaintext Message with EDS . . . . . . . . . . . . . . . 35 142 13.3. Encrypted Message . . . . . . . . . . . . . . . . . . . 35 143 13.4. Signed Message . . . . . . . . . . . . . . . . . . . . . 37 144 13.5. Signed and Encrypted Message . . . . . . . . . . . . . . 38 145 14. Appendix B: DARE Container Examples and Test Vectors . . . . 39 146 14.1. Simple container . . . . . . . . . . . . . . . . . . . . 39 147 14.2. Payload and chain digests . . . . . . . . . . . . . . . 40 148 14.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . 41 149 14.4. Signed container . . . . . . . . . . . . . . . . . . . . 43 150 14.5. Encrypted container . . . . . . . . . . . . . . . . . . 44 151 15. Appendix C: Previous Frame Function . . . . . . . . . . . . . 46 152 16. Appendix D: Outstanding Issues . . . . . . . . . . . . . . . 46 153 17. References . . . . . . . . . . . . . . . . . . . . . . . . . 47 154 17.1. Normative References . . . . . . . . . . . . . . . . . . 47 155 17.2. Informative References . . . . . . . . . . . . . . . . . 48 156 17.3. URIs . . . . . . . . . . . . . . . . . . . . . . . . . . 49 157 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 49 159 1. Introduction 161 This document describes the Data At Rest Encryption (DARE) Envelope 162 and Container Syntax. The DARE Envelope syntax is used to digitally 163 sign, digest, authenticate, or encrypt arbitrary message content. 164 The DARE Container syntax describes an append-only sequence of data 165 frames, each containing a DARE Envelope that supports efficient 166 incremental signature and encryption. 168 The DARE Envelope Syntax is based on a subset of the JSON Web 169 Signature [RFC7515] and JSON Web Encryption [RFC7516] standards and 170 shares many fields and semantics. The processing model and data 171 structures have been streamlined to remove alternative means of 172 specifying the same content and to enable multiple data sequences to 173 be signed and encrypted under a single master encryption key without 174 compromise to security. 176 A DARE Envelope consists of a Header, Payload and an optional 177 Trailer. To enable single pass encoding and decoding, the Header 178 contains all the information required to perform cryptographic 179 processing of the Payload and authentication data (digest, MAC, 180 signature values) MAY be deferred to the Trailer section. 182 A DARE Container is an append-only log format consisting of a 183 sequence of frames. Cryptographic enhancements (signature, 184 encryption) may be applied to individual frames or to sets of frames. 185 Thus, a single key exchange may be used to provide a master key to 186 encrypt multiple frames and a single signature may be used to 187 authenticate all the frames in the container up to and including the 188 frame in which the signature is presented. 190 The DARE Envelope syntax may be used either as a standalone 191 cryptographic message syntax or as a means of presenting a single 192 DARE Container frame together with the complete cryptographic context 193 required to verify the contents and decrypt them. 195 1.1. Encryption and Integrity 197 A key innovation in the DARE Envelope Syntax is the separation of key 198 exchange and data encryption operations so that a Master Key (MK) 199 established in a single exchange to be applied to multiple data 200 sequences. This means that a single public key operation MAY be used 201 to encrypt and/or authenticate multiple parts of the same DARE 202 Envelope or multiple frames in a DARE Container. 204 To avoid reuse of the key and to avoid the need to communicate 205 separate IVs, each octet sequence is encrypted under a different 206 encryption key (and IV if required) derived from the Master Key by 207 means of a salt that is unique for each octet sequence that is 208 encrypted. The same approach is used to generate keys for 209 calculating a MAC over the octet sequence if required. This approach 210 allows encryption and integrity protections to be applied to the 211 envelope payload, to header or trailer fields or to application 212 defined Enhanced Data Sequences in the header or trailer. 214 1.1.1. Key Exchange 216 Traditional cryptographic containers describe the application of a 217 single key exchange to encryption of a single octet sequence. 218 Examples include PCKS#7/CMS [RFC2315] , OpenPGP [RFC4880] and JSON 219 Web Encryption [RFC7516] . 221 To encrypt data using RSA, the encoder first generates a random 222 encryption key and initialization vector (IV). The encryption key is 223 encrypted under the public key of each recipient to create a per- 224 recipient decryption entry. The encryption key, plaintext and IV are 225 used to generate the ciphertext (figure 1). 227 [[This figure is not viewable in this format. The figure is 228 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 229 dare.html [2].]] 231 Monolithic Key Exchange and Encrypt 233 This approach is adequate for the task of encrypting a single octet 234 stream. It is less than satisfactory when encrypting multiple octet 235 streams or very long streams for which a rekeying operation is 236 desirable. 238 In the DARE approach, key exchange and key derivation are separate 239 operations and keys MAY be derived for encryption or integrity 240 purposes or both. A single key exchange MAY be used to derive keys 241 to apply encryption and integrity enhancements to multiple data 242 sequences. 244 The DARE key exchange begins with the same key exchange used to 245 produce the CEK in JWE but instead of using the CEK to encipher data 246 directly, it is used as one of the inputs to a Key Derivation 247 Function (KDF) that is used to derive parameters for each block of 248 data to be encrypted. To avoid the need to introduce additional 249 terminology, the term 'CEK' is still used to describe the output of 250 the key agreement algorithm (including key unwrapping if required) 251 but it is more appropriately described as a Master Key (figure 2). 253 [[This figure is not viewable in this format. The figure is 254 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 255 dare.html [3].]] 257 Exchange of Master Key 259 A Master Key may be used to encrypt any number of data items. Each 260 data item is encrypted under a different encryption key and IV (if 261 required). This data is derived from the Master Key using the HKDF 262 function [RFC5869] using a different salt for each data item and 263 separate info tags for each cryptographic function (figure 3). 265 [[This figure is not viewable in this format. The figure is 266 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 267 dare.html [4].]] 269 Data item encryption under Master Key and per-item salt. 271 This approach to encryption offers considerably greater flexibility 272 allowing the same format for data item encryption to be applied at 273 the transport, message or field level. 275 1.1.2. Data Erasure 277 Each encrypted DARE Envelope specifies a unique Master Salt value of 278 at least 128 bits which is used to derive the salt values used to 279 derive cryptographic keys for the envelope payload and annotations. 281 Erasure of the Master Salt value MAY be used to effectively render 282 the envelope payload and annotations undecipherable without altering 283 the envelope payload data. The work factor for decryption will be 284 O(2^128) even if the decryption key is compromised. 286 1.2. Signature 288 As with encryption, DARE Envelope signatures MAY be applied to an 289 individual envelope or a sequence of envelope. 291 1.2.1. Signing Individual Plaintext Envelopes 293 When an individual plaintext envelope is signed, the digest value 294 used to create the signature is calculated over the binary value of 295 the payload data. That is, the value of the payload before the 296 encoding (Base-64, JSON-B) is applied. 298 1.2.2. Signing Individual Encrypted Envelopes 300 When an individual plaintext envelope is signed, the digest value 301 used to create the signature is calculated over the binary value of 302 the payload data. That is, the value of the payload after encryption 303 but before the encoding (Base-64, JSON-B) is applied. 305 Use of signing and encryption in combination presents the risk of 306 subtle attacks depending on the order in which signing and encryption 307 take place [Davis2001] . 309 Na?ve approaches in which an envelope is encrypted and then signed 310 present the possibility of a surreptitious forwarding attack. For 311 example, Alice signs an envelope and sends it to Mallet who then 312 strips off Alice's signature and sends the envelope to Bob. 314 Na?ve approaches in which an envelope is signed and then encrypted 315 present the possibility of an attacker claiming authorship of a 316 ciphertext. For example, Alice encrypts a ciphertext for Bob and 317 then signs it. Mallet then intercepts the envelope and sends it to 318 Bob. 320 While neither attack is a concern in all applications, both attacks 321 pose potential hazards for the unwary and require close inspection of 322 application protocol design to avoid exploitation. 324 To prevent these attacks, each signature on an envelope that is 325 signed and encrypted MUST include a witness value that is calculated 326 by applying a MAC function to the signature value as described in 327 section XXX. 329 1.2.3. Signing sequences of envelopes 331 To sign multiple envelopes with a single signature, we first 332 construct a Merkle tree of the envelope payload digest values and 333 then sign the root of the Merkle tree. 335 [This is not yet implemented but will be soon.] 337 1.3. Container 339 DARE Container is a message and file syntax that allows a sequence of 340 data frames to be represented with cryptographic integrity, 341 signature, and encryption enhancements to be constructed in an append 342 only format. 344 The format is designed to meet the requirements of a wide range of 345 use cases including: 347 o Recording transactions in persistent storage. 349 o Synchronizing transaction logs between hosts. 351 o File archive. 353 o Message spool. 355 o Signing and encrypting single data items. 357 o Incremental encryption and authentication of server logs. 359 1.3.1. Container Format 361 A Container consists of a sequence of variable length Frames. Each 362 frame consists of a forward length indicator, the framed data and a 363 reverse length indicator. The reverse length indicator is written 364 out backwards allowing the length and thus the frame to be read in 365 the reverse direction: 367 [[This figure is not viewable in this format. The figure is 368 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 369 dare.html [5].]] 371 JBCD Bidirectional Frame 373 Each frame contains a single DARE Envelope consisting of a Header, 374 Payload and Trailer (if required). The first frame in a container 375 describes the container format options and defaults. These include 376 the range of encoding options for frame metadata supported and the 377 container profiles to which the container conforms. 379 All internal data formats support use of pointers of up to 64 bits 380 allowing containers of up to 18 exabytes to be written. 382 Five container types are currently specified: 384 Simple The container does not provide any index or content integrity 385 checks. 387 Tree Frame headers contain entries that specify the start position 388 of previous frames at the apex of the immediately enclosing binary 389 tree. This enables efficient random access to any frame in the 390 file. 392 Digest Each frame trailer contains a PayloadDigest field. 393 Modification of the payload will cause verification of the 394 PayloadDigest value to fail on that frame. 396 Chain Each frame trailer contains PayloadDigest and ChainDigest 397 fields allowing modifications to the payload data to be detected. 398 Modification of the payload will cause verification of the 399 PayloadDigest value to fail on that frame and verification of the 400 ChainDigest value to fail on all subsequent frames. 402 Merkle Tree Frame headers contain entries that specify the start 403 position of previous frames at the apex of the immediately 404 enclosing binary tree. Frame Trailers contain 405 TreeDigestPartial and TreeDigestFinal entries forming a Merkle 406 digest tree. 408 1.3.2. Write 410 In normal circumstances, Containers are written as an append only 411 log. As with Envelopes, integrity information (payload digest, 412 signatures) is written to the entry trailer. Thus, large payloads 413 may be written without the need to buffer the payload data provided 414 that the content length is known in advance. 416 Should exceptional circumstances require, Container entries MAY be 417 erased by overwriting the Payload and/or parts of the Header content 418 without compromising the ability to verify other entries in the 419 container. If the entry Payload is encrypted, it is sufficient to 420 erase the container salt value to render the container entry 421 effectively inaccessible (though recovery might still be possible if 422 the original salt value can be recovered from the storage media. 424 1.3.3. Encryption and Authentication 426 Frame payloads and associated attributes MAY be encrypted and/or 427 authenticated in the same manner as Envelopes. 429 Incremental encryption is supported allowing encryption parameters 430 from a single public key exchange operation to be applied to encrypt 431 multiple frames. The public key exchange information is specified in 432 the first encrypted frame and subsequent frames encrypted under those 433 parameters specify the location at which the key exchange information 434 is to be found by means of the ExchangePosition field which MUST 435 specify a location that is earlier in the file. 437 To avoid cryptographic vulnerabilities resulting from key re-use, the 438 DARE key exchange requires that each encrypted sequence use an 439 encryption key and initialization vector derived from the master key 440 established in the public key exchange by means of a unique salt. 442 Each Envelope and by extension, each Container frame MUST specify a 443 unique salt value of at least 128 bits. Since the encryption key is 444 derived from the salt value by means of a Key Derivation Function, 445 erasure of the salt MAY be used as a means of rendering the payload 446 plaintext value inaccessible without changing the payload value. 448 1.3.4. Integrity and Signature 450 Signatures MAY be applied to a payload digest, the final digest in a 451 chain or tree. The chain and tree digest modes allow a single 452 signature to be used to authenticate all frame payloads in a 453 container. 455 The tree signature mode is particularly suited to applications such 456 as file archives as it allows files to be verified individually 457 without requiring the signer to sign each individually. Furthermore, 458 in applications such as code signing, it allows a single signature to 459 be used to verify both the integrity of the code and its membership 460 of the distribution. 462 As with DARE Envelope, the signature mechanism does not specify the 463 interpretation of the signature semantics. The presence of a 464 signature demonstrates that the holder of the private key applied it 465 to the specified digest value but not their motive for doing so. 466 Describing such semantics is beyond the scope of this document and is 467 deferred to future work. 469 1.3.5. Redaction 471 The chief disadvantage of using an append-only format is that 472 containers only increase in size. In many applications, much of the 473 data in the container becomes redundant or obsolete and a process 474 analogous to garbage collection is required. This process is called 475 redaction. 477 The simplest method of redaction is to create a new container and 478 sequentially copy each entry from the old container to the new, 479 discarding redundant frames and obsolete header information. 481 For example, partial index records may be consolidated into a single 482 index record placed in the last frame of the container. Unnecessary 483 signature and integrity data may be discarded and so on. 485 While redaction could in principle be effected by moving data in- 486 place in the existing container, supporting this approach in a robust 487 fashion is considerably more complex as it requires backward 488 references in subsequent frames to be overridden as each frame is 489 moved. 491 1.3.6. Alternative approaches 493 Many file proprietary formats are in use that support some or all of 494 these capabilities but only a handful have public, let alone open, 495 standards. DARE Container is designed to provide a superset of the 496 capabilities of existing message and file syntaxes, including: 498 o Cryptographic Message Syntax [RFC5652] defines a syntax used to 499 digitally sign, digest, authenticate, or encrypt arbitrary message 500 content. 502 o The.ZIP File Format specification [ZIPFILE] developed by Phil 503 Katz. 505 o The BitCoin Block chain [BLOCKCHAIN] . 507 o JSON Web Encryption and JSON Web Signature 509 Attempting to make use of these specifications in a layered fashion 510 would require at least three separate encoders and introduce 511 unnecessary complexity. Furthermore, there is considerable overlap 512 between the specifications providing multiple means of achieving the 513 same ends, all of which must be supported if decoders are to work 514 reliably. 516 1.3.7. Efficiency 518 Every data format represents a compromise between different concerns, 519 in particular: 521 Compactness The space required to record data in the encoding. 523 Memory Overhead The additional volatile storage (RAM) required to 524 maintain indexes etc. to support efficient retrieval operations. 526 Number of Operations The number of operations required to retrieve 527 data from or append data to an existing encoded sequence. 529 Number of Disk Seek Operations Optimizing the response time of 530 magnetic storage media to random access read requests has 531 traditionally been one of the central concerns of database design. 532 The DARE Container format is designed to the assumption that this 533 will cease to be a concern as solid state media replaces magnetic. 535 While the cost of storage of all types has declined rapidly over the 536 past decades, so has the amount of data to be stored. DARE Container 537 represents a pragmatic balance of these considerations for current 538 technology. In particular, since payload volumes are likely to be 539 very large, memory and operational efficiency are considered higher 540 priorities than compactness. 542 2. Definitions 544 2.1. Related Specifications 546 The DARE Envelope and Container formats are based on the following 547 existing standards and specifications. 549 Object serialization The JSON-B [draft-hallambaker-jsonbcd] encoding 550 is used for object serialization. This encoding is an extension 551 of the JavaScript Object Notation (JSON) [RFC7159] . 553 Message syntax The cryptographic processing model is based on JSON 554 Web Signature (JWS) [RFC7515] , JSON Web Encryption (JWE) 555 [RFC7516] and JSON Web Key (JWK) [RFC7517] . 557 Cryptographic primitives. The HMAC-based Extract-and-Expand Key 558 Derivation Function [RFC5869] and Advanced Encryption Standard 559 (AES) Key Wrap with Padding Algorithm [RFC3394] are used. 561 The Uniform Data Fingerprint method of presenting data digests is 562 used for key identifiers and other purposes 563 [draft-hallambaker-mesh-udf] . 565 Cryptographic algorithms The cryptographic algorithms and 566 identifiers described in JSON Web Algorithms (JWA) [RFC7518] are 567 used together with additional algorithms as defined in the JSON 568 Object Signing and Encryption IANA registry [IANAJOSE] . 570 2.2. Requirements Language 572 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 573 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 574 document are to be interpreted as described in [RFC2119] . 576 2.3. Defined terms 578 The terms "Authentication Tag", "Content Encryption Key", "Key 579 Management Mode", "Key Encryption", "Direct Key Agreement", "Key 580 Agreement with Key Wrapping" and "Direct Encryption" are defined in 581 the JWE specification [RFC7516] . 583 The terms "Authentication", "Ciphertext", "Digital Signature", 584 "Encryption", "Initialization Vector (IV)", "Message Authentication 585 Code (MAC)", "Plaintext" and "Salt" are defined by the Internet 586 Security Glossary, Version 2 [RFC4949] . 588 Annotated Envelope A DARE Envelope that contains an Annotations 589 field with at least one entry. 591 Authentication Data A Message Authentication Code or authentication 592 tag. 594 Complete Envelope A DARE envelope that contains the key exchange 595 information necessary for the intended recipient(s) to decrypt it. 597 Detached Envelope A DARE envelope that does not contain the key 598 exchange information necessary for the intended recipient(s) to 599 decrypt it. 601 Encryption Context The master key, encryption algorithms and 602 associated parameters used to generate a set of one or more 603 enhanced data sequences. 605 Encoded data sequence (EDS) A sequence consisting of a salt, content 606 data and authentication data (if required by the encryption 607 context). 609 Enhancement Applying a cryptographic operation to a data sequence. 610 This includes encryption, authentication and both at the same 611 time. 613 Generator The party that generates a DARE envelope. 615 Group Encryption Key A key used to encrypt data to be read by a 616 group of users. This is typically achieved by means of some form 617 of proxy re-encryption or distributed key generation. 619 Group Encryption Key Identifier A key identifier for a group 620 encryption key. 622 Master Key (MK) The master secret from which keys are derived for 623 authenticating enhanced data sequences. 625 Recipient Any party that receives and processes at least some part 626 of a DARE envelope. 628 Related Envelope A set of DARE envelopes that share the same key 629 exchange information and hence the same Master Key. 631 Uniform Data Fingerprint (UDF) The means of presenting the result of 632 a cryptographic digest function over a data sequence and content 633 type identifier specified in the Uniform Data Fingerprint 634 specification [draft-hallambaker-mesh-udf] 636 3. DARE Envelope Architecture 638 A DARE Envelope is a sequence of three parts: 640 Header A JSON object containing information a reader requires to 641 begin processing the envelope. 643 Payload An array of octets. 645 Trailer A JSON object containing information calculated from the 646 envelope payload. 648 For example, the following sequence is a JSON encoded Envelope with 649 an empty header, a payload of zero length and an empty trailer: 651 [ {}, "", {} ] 653 DARE Envelopes MAY be encoded using JSON serialization or a binary 654 serialization for greater efficiency. 656 JSON Offers compatibility with applications and libraries that 657 support JSON. Payload data is encoded using Base64 incurring a 658 33% overhead. 660 JSON-B A superset of JSON encoding that permits binary data to be 661 encoded as a sequence of length-data segments. This avoids the 662 Base64 overhead incurred by JSON encoding. Since JSON-B is a 663 superset of JSON encoding, an application can use a single decoder 664 for either format. 666 JSON-C A superset of JSON-C which provides additional efficiency by 667 allowing field tags and other repeated string data to be encoded 668 by reference to a dictionary. Since JSON-C is a superset of JSON 669 and JSON-B encodings, an application can use a single decoder for 670 all three formats. 672 DARE Envelope processors MUST support JSON serialization and SHOULD 673 support JSON-B serialization. 675 3.1. Processing Considerations 677 The DARE Envelope Syntax supports single pass encoding and decoding 678 without buffering of data. All the information required to begin 679 processing a DARE envelope (key agreement information, digest 680 algorithms), is provided in the envelope header. All the information 681 that is derived from envelope processing (authentication codes, 682 digest values, signatures) is presented in the envelope trailer. 684 The choice of envelope encoding does not affect the semantics of 685 envelope processing. A DARE Envelope MAY be reserialized under the 686 same serialization or converted from any of the specified 687 serialization to any other serialization without changing the 688 semantics or integrity properties of the envelope. 690 3.2. Content Metadata and Annotations 692 A header MAY contain header fields describing the payload content. 693 These include: 695 ContentType Specifies the IANA Media Type [RFC6838] . 697 Annotations A list of Encoded Data Sequences that provide 698 application specific annotations to the envelope. 700 For example, consider the following mail message: 702 From: Alice@example.com 703 To: bob@example.com 704 Subject: TOP-SECRET Product Launch Today! 706 The CEO told me the product launch is today. Tell no-one! 707 Existing encryption approaches require that header fields such as the 708 subject line be encrypted with the body of the message or not 709 encrypted at all. Neither approach is satisfactory. In this 710 example, the subject line gives away important information that the 711 sender probably assumed would be encrypted. But if the subject line 712 is encrypted together with the message body, a mail client must 713 retrieve at least part of the message body to provide a 'folder' 714 view. 716 The plaintext form of the equivalent DARE Message encoding is: 718 [{ 719 "cty":"application/example-mail", 720 "Annotations":["iAEBiBdGcm9tOiBBbGljZUBleGFtcGxlLmNvbYgA", 721 "iAECiBNUbzogYm9iQGV4YW1wbGUuY29tiAA", 722 "iAEDiClTdWJqZWN0OiBUT1AtU0VDUkVUIFByb2R1Y3QgTGF1bmNoIFRvZGF5 723 IYgA" 724 ]}, 725 "VGhlIENFTyB0b2xkIG1lIHRoZSBwcm9kdWN0IGxhdW5jaCBpcyB0b2RheS4gVGVs 726 bCBuby1vbmUh" 727 ] 729 This contains the same information as before but the mail message 730 headers are now presented as a list of Encoded Data Sequences. 732 3.3. Encoded Data Sequence 734 An encoded data sequence (EDS) is a sequence of octets that encodes a 735 data sequence according to cryptographic enhancements specified in 736 the context in which it is presented. An EDS MAY be encrypted and 737 MAY be authenticated by means of a MAC. The keys and other 738 cryptographic parameters used to apply these enhancements are derived 739 from the cryptographic context and a Salt prefix specified in the EDS 740 itself. 742 An EDS sequence contains exactly three binary fields encoded in 743 JSON-B serialization as follows: 745 Salt Prefix A sequence of octets used to derive the encryption key, 746 Initialization Vector and MAC key as required. 748 Body The plaintext or encrypted content. 750 Authentication Tag The authentication code value in the case that 751 the cryptographic context specifies use of authenticated 752 encryption or a MAC, otherwise is a zero-length field. 754 Requiring all three fields to be present, even in cases where they 755 are unnecessary simplifies processing at the cost of up to six 756 additional data bytes. 758 The encoding of the 'From' header of the previous example as a 759 plaintext EDS is as follows: 761 88 01 762 01 763 88 17 764 46 72 6f 6d 3a 20 41 6c 69 63 65 40 65 78 61 6d 765 70 6c 65 2e 63 6f 6d 766 88 00 768 3.4. Encryption and Integrity 770 Encryption and integrity protections MAY be applied to any DARE 771 Envelope Payload and Annotations. 773 The following is an encrypted version of the message shown earlier. 774 The payload and annotations have both increased in size as a result 775 of the block cipher padding. The header now includes Recipients and 776 Salt fields to enable the content to be decoded. 778 [{ 779 "enc":"A256CBC", 780 "Salt":"1yar8uEuI4EuncLCO3-LKA", 781 "cty":"application/example-mail", 782 "Annotations":["iAEBiCCKgGLHHsxdpvGblD8DK0T-KwKzBgAgxn-KIysGeu8c 783 JA", 784 "iAECiCBXxyGLLaAwp2Ors9xUwe-JCJlh8AfWj0bBmEdh2dvTcg", 785 "iAEDiDBPFSF5bjzBBso_CKuu-4pciBOccVJRBuuQOkhBba9c3h9z0WGCwvZB 786 5gXJyhBQH8k" 787 ], 788 "recipients":[{ 789 "kid":"MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 790 "epk":{ 791 "PublicKeyECDH":{ 792 "crv":"Ed25519", 793 "Public":"CLChDsgiK_TRwSbNRpRhpWz_ZKU9lPhP_Uol7Su8FAo"}}, 794 "wmk":"Hq37806YKSo98_66t02Zw6FTNEtz9mjE923IP5HtLjvsSW5lpT40 795 cQ"} 796 ]}, 797 "devgznyt0uIljanjJ7CexSww6KxLF_V_blMXS9jyx7UwD8pvaQV-wpiGHz_Od0aS 798 07AON1He9Zp-KjnlDLsMyg" 799 ] 801 3.4.1. Key Exchange 803 The DARE key exchange is based on the JWE key exchange except that 804 encryption modes are intentionally limited and the output of the key 805 exchange is the DARE Master Key rather than the Content Encryption 806 Key. 808 A DARE Key Exchange MAY contain any number of Recipient entries, each 809 providing a means of decrypting the Master Key using a different 810 private key. 812 If the Key Exchange mechanism supports message recovery, Direct Key 813 Agreement is used, in all other cases, Key Wrapping is used. 815 This approach allows envelopes with one intended recipient to be 816 handled in the exact same fashion as envelopes with multiple 817 recipients. While this does require an additional key wrapping 818 operation, that could be avoided if an envelope has exactly one 819 intended recipient, this is offset by the reduction in code 820 complexity. 822 If the key exchange algorithm does not support message recovery (e.g. 823 Diffie Hellman and Elliptic Curve Diffie-Hellman), the HKDF Extract- 824 and-Expand Key Derivation Function is used to derive a master key 825 using the following info tag: 827 "dare-master" [64 61 72 65 2d 6d 61 73 74 65 72] Key derivation info 828 field used when deriving a master key from the output of a key 829 exchange. 831 The master key length is the maximum of the key size of the 832 encryption algorithm specified by the key exchange header, the key 833 size of the MAC algorithm specified by the key exchange header (if 834 used) and 256. 836 3.4.2. Key Identifiers 838 The JWE/JWS specifications define a kid field for use as a key 839 identifier but not how the identifier itself is constructed. All 840 DARE key identifiers are either UDF key fingerprints 841 [draft-hallambaker-mesh-udf] or Mesh/Recrypt Group Key Identifiers. 843 A UDF fingerprint is formed as the digest of an IANA content type and 844 the digested data. A UDF key fingerprint is formed with the content 845 type application/pkix-keyinfo and the digested data is the ASN.1 DER 846 encoded PKIX certificate keyInfo sequence for the corresponding 847 public key. 849 A Group Key Identifier has the form @. Where 850 is a UDF key fingerprint and is the DNS 851 address of a service that provides the encryption service to support 852 decryption by group members. 854 3.4.3. Salt Derivation 856 A Master Salt is a sequence of 16 or more octets that is specified in 857 the Salt field of the header. 859 The Master Salt is used to derive salt values for the envelope 860 payload and associated encoded data sequences as follows. 862 Payload Salt = Master Salt 864 EDS Salt = Concatenate (Payload Salt Prefix, Master Salt) 866 Encoders SHOULD NOT generate salt values that exceed 1024 octets. 868 The salt value is opaque to the DARE encoding but MAY be used to 869 encode application specific semantics including: 871 o Frame number to allow reassembly of a data sequence split over a 872 sequence of envelopes which may be delivered out of order. 874 o Transmit the Master Key in the manner of a Kerberos ticket. 876 o Identify the Master Key under which the Enhanced Data Sequence was 877 generated. 879 o Enable access to the plaintext to be eliminated by erasure of the 880 encryption key. 882 For data erasure to be effective, the salt MUST be constructed so 883 that the difficulty of recovering the key is sufficiently high that 884 it is infeasible. For most purposes, a salt with 128 bits of 885 appropriately random data is sufficient. 887 3.4.4. Key Derivation 889 Encryption and/or authentication keys are derived from the Master Key 890 using a Extract-and-Expand Key Derivation Function as follows: 892 1. The Master Key and salt value are used to extract the PRK 893 (pseudorandom key) 895 2. The PRK is used to derive the algorithm keys using the 896 application specific information input for that key type. 898 The application specific information inputs are: 900 "dare-encrypt" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 901 encryption or encryption with authentication key. 903 "dare-iv" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 904 initialization vector. 906 "dare-mac" [dare-mac] To generate a Message Authentication Code key. 908 3.5. Signature 910 While encryption and integrity enhancements can be applied to any 911 part of a DARE Envelope, signatures are only applied to payload 912 digest values calculated over one or more envelope payloads. 914 The payload digest value for an envelope is calculated over the 915 binary payload data. That is, after any encryption enhancement has 916 been applied but before the envelope encoding is applied. This 917 allows envelopes to be converted from one encoding to another without 918 affecting signature verification. 920 Single Payload The signed value is the payload digest of the 921 envelope payload. 923 Multiple Payload. The signed value is the root of a Merkle Tree in 924 which the payload digest of the envelope is one of the leaves. 926 Verification of a multiple payload signature naturally requires the 927 additional digest values required to construct the Merkle Tree. 928 These are provided in the Trailer in a format that permits multiple 929 signers to reference the same tree data. 931 3.6. Algorithms 933 3.6.1. Field: kwd 935 The key wrapping and derivation algorithms. 937 Since the means of public key exchange is determined by the key 938 identifier of the recipient key, it is only necessary to specify the 939 algorithms used for key wrapping and derivation. 941 The default (and so far only) algorithm is kwd-aes-sha2-256-256. 943 Advanced Encryption Standard (AES) Key Wrap with Padding Algorithm 944 [RFC3394] is used to wrap the Master Exchange Key. AES 256 is used. 946 HMAC-based Extract-and-Expand Key Derivation Function [RFC5869] is 947 used for key derivation. SHA-2-256 is used for the hash function. 949 4. DARE Container Architecture 951 4.1. Container Navigation 953 Three means of locating frames in a container are supported: 955 Sequential Access frames sequentially starting from the start or the 956 end of the container. 958 Binary search Access any container frame by frame number in 959 O(log_2(n)) time by means of a binary tree constructed while the 960 container is written. 962 Index Access and container frame by frame number or by key by means 963 of an index record. 965 All DARE Containers support sequential access. Only tree and Merkle 966 tree containers support binary search access. An index frame MAY be 967 written appended to any container and provides O(1) access to any 968 frame listed in the index. 970 Two modes of compilation are considered: 972 Monolithic Frames are added to the container in a single operation, 973 e.g. file archives, 975 Incremental Additional frames are written to the container at 976 various intervals after it was originally created, e.g. server 977 logs, message spools. 979 In the monolithic mode, navigation requirements are best met by 980 writing an index frame to the end of the container when it is 981 complete. It is not necessary to construct a binary search tree 982 unless a Merkle tree integrity check is required. 984 In the incremental mode, Binary search provides an efficient means of 985 locating frames by frame number but not by key. Writing a complete 986 index to the container every m write operations provides O(m) search 987 access but requires O(n^2) storage. 989 Use of partial indexes provides a better compromise between speed and 990 efficiency. A partial index is written out every m frames where m is 991 a power of two. A complete index is written every time a binary tree 992 apex record is written. This approach provides for O(log_2(n)) 993 search with incremental compilation with approximately double the 994 overhead of the monolithic case. 996 4.1.1. Tree 998 Binary search is supported by means of the TreePosition parameter 999 specified in the FrameHeader. This parameter specifies the value of 1000 the immediately preceding apex. 1002 Calculation of the immediately preceding apex is most easily 1003 described by representing the array index in binary with base of 1 1004 (rather than 0). An array index that is a power of 2 (2, 4, 8, 16, 1005 etc.) will be the apex of a complete tree. Every other array index 1006 has the value of the sum of a set of powers of 2 and the immediately 1007 preceding apex will be the value of the next smallest power of 2 in 1008 the sum. 1010 For example, to find the immediately preceding apex for frame 5, we 1011 add 1 to get 6. 6 = 4 + 2, so we ignore the 2 and the preceding frame 1012 is 4. 1014 The values of Tree Position are shown for the first 8 frames in 1015 figure xx below: 1017 [[This figure is not viewable in this format. The figure is 1018 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 1019 dare.html [6].]] 1021 Merkle Tree Integrity check 1023 An algorithm for efficiently calculating the immediately preceding 1024 apex is provided in Appendix C. 1026 4.1.2. Position Index 1028 Contains a table of frame number, position pairs pointing to prior 1029 locations in the file. 1031 4.1.3. Metadata Index 1033 Contains a list of IndexMeta entries. Each entry contains a metadata 1034 description and a list of frame indexes (not positions) of frames 1035 that match the description. 1037 4.2. Integrity Mechanisms 1039 Frame sequences in a DARE container MAY be protected against a frame 1040 insertion attack by means of a digest chain, a binary Merkle tree or 1041 both. 1043 4.2.1. Digest Chain calculation 1045 A digest chain is simple to implement but can only be verified if the 1046 full chain of values is known. Appending a frame to the chain has 1047 O(1) complexity but verification has O(n) complexity: 1049 [[This figure is not viewable in this format. The figure is 1050 available at http://mathmesh.com/Documents/draft-hallambaker-mesh- 1051 dare.html [7].]] 1053 Hash chain integrity check 1055 The value of the chain digest for the first frame (frame 0) is 1056 H(H(null)+H(Payload_0)), where null is a zero length octet sequence 1057 and payloadn is the sequence of payload data bytes for frame n 1059 The value of the chain digest for frame n is H(H(Payload_n-1 1060 +H(Payloadn)), where A+B stands for concatenation of the byte 1061 sequences A and B. 1063 4.2.2. Binary Merkle tree calculation 1065 The tree index mechanism describe earlier may be used to implement a 1066 binary Merkle tree. The value TreeDigest specifies the apex value of 1067 the tree for that node. 1069 Appending a frame to the chain has O(log_2 (n)) complexity provided 1070 that the container format supports at least the binary tree index. 1071 Verifying a chain has O(log_2 (n)) complexity, provided that the set 1072 of necessary digest inputs is known. 1074 To calculate the value of the tree digest for a node, we first 1075 calculate the values of all the sub trees that have their apex at 1076 that node and then calculate the digest of that value and the 1077 immediately preceding local apex. 1079 4.2.3. Signature 1081 Payload data MAY be signed using a JWS [RFC7515] as applied in the 1082 Envelope. 1084 Signatures are specified by the Signatures parameter in the content 1085 header. The data that the signature is calculated over is defined by 1086 the typ parameter of the Signature as follows. 1088 Payload The value of the PayloadDigest parameter 1090 Chain The value of the ChainDigest parameter 1092 Tree The value of the TreeDigestFinal parameter 1094 If the typ parameter is absent, the value Payload is implied. 1096 A frame MAY contain multiple signatures created with the same signing 1097 key and different typ values. 1099 The use of signatures over chain and tree digest values permit 1100 multiple frames to be validated using a single signature verification 1101 operation. 1103 5. DARE Schema 1105 A DARE Envelope consists of a Header, an Enhanced Data Sequence (EDS) 1106 and an optional trailer. This section describes the JSON data fields 1107 used to construct headers, trailers and complete messages. 1109 Wherever possible, fields from JWE, JWS and JWK have been used. In 1110 these cases, the fields have the exact same semantics. Note however 1111 that the classes in which these fields are presented have different 1112 structure and nesting. 1114 5.1. Message Classes 1116 A DARE Message contains a single DAREMessageSequence in either the 1117 JSON or Compact serialization as directed by the protocol in which it 1118 is applied. 1120 5.1.1. Structure: DareEnvelopeSequence 1122 A DARE Message containing Header, EDS and Trailer in JSON object 1123 encoding. Since a DAREMessage is almost invariably presented in JSON 1124 sequence or compact encoding, use of the DAREMessage subclass is 1125 preferred. 1127 Although a DARE Message is functionally an object, it is serialized 1128 as an ordered sequence. This ensures that the message header field 1129 will always precede the body in a serialization, this allowing 1130 processing of the header information to be performed before the 1131 entire body has been received. 1133 Header: DareHeader (Optional) The message header. May specify the 1134 key exchange data, pre-signature or signature data, cloaked 1135 headers and/or encrypted data sequences. 1137 Body: Binary (Optional) The message body 1139 Trailer: DareTrailer (Optional) The message trailer. If present, 1140 this contains the signature. 1142 5.2. Header and Trailer Classes 1144 A DARE Message sequence MUST contain a (possibly empty) DAREHeader 1145 and MAY contain a DARETrailer. 1147 5.2.1. Structure: DareTrailer 1149 A DARE Message Trailer 1151 Signatures: DareSignature [0..Many] A list of signatures. A message 1152 trailer MUST NOT contain a signatures field if the header contains 1153 a signatures field. 1155 SignedData: Binary (Optional) Contains a DAREHeader object 1157 PayloadDigest: Binary (Optional) If present, contains the digest of 1158 the Payload. 1160 ChainDigest: Binary (Optional) If present, contains the digest of 1161 the PayloadDigest values of this frame and the frame immediately 1162 preceding. 1164 TreeDigest: Binary (Optional) If present, contains the Binary Merkle 1165 Tree digest value. 1167 5.2.2. Structure: DareHeader 1169 Inherits: DareTrailer 1171 A DARE Message Header. Since any field that is present in a trailer 1172 MAY be placed in a header instead, the message header inherits from 1173 the trailer. 1175 EncryptionAlgorithm: String (Optional) The encryption algorithm as 1176 specified in JWE 1178 DigestAlgorithm: String (Optional) Digest Algorithm. If specified, 1179 tells decoder that the digest algorithm is used to construct a 1180 signature over the message payload. 1182 Salt: Binary (Optional) Salt value used to derrive cryptographic 1183 parameters for the content data. 1185 Malt: Binary (Optional) Hash of the Salt value used to derrive 1186 cryptographic parameters for the content data. This field SHOULD 1187 NOT be present if the Salt field is present. It is used to allow 1188 the salt value to be erased (thus rendering the payload content 1189 irrecoverable) without affecting the ability to calculate the 1190 payload digest value. 1192 Signed: Binary (Optional) Contains signed headers. 1194 Cloaked: Binary (Optional) If present in a header or trailer, 1195 specifies an encrypted data block containing additional header 1196 fields whose values override those specified in the message and 1197 context headers. 1199 When specified in a header, a cloaked field MAY be used to conceal 1200 metadata (content type, compression) and/or to specify an 1201 additional layer of key exchange. That applies to both the 1202 Message body and to headers specified within the cloaked header. 1204 Processing of cloaked data is described in... 1206 ContentType: String (Optional) The content type field as specified 1207 in JWE 1209 EDSS: Binary [0..Many] If present, the Annotations field contains a 1210 sequence of Encrypted Data Segments encrypted under the message 1211 Master Key. The interpretation of these fields is application 1212 specific. 1214 Signers: DareSigner [0..Many] A list of 'presignature' 1216 Recipients: DareRecipient [0..Many] A list of recipient key exchange 1217 information blocks. 1219 UniqueID: String (Optional) Unique object identifier 1221 Filename: String (Optional) The original filename under which the 1222 data was stored. 1224 Event: String (Optional) Operation on the header 1226 Labels: String [0..Many] List of labels that are applied to the 1227 payload of the frame. 1229 KeyValues: KeyValue [0..Many] List of key/value pairs describing the 1230 payload of the frame. 1232 5.3. Cryptographic Data 1234 DARE Message uses the same fields as JWE and JWS but with different 1235 structure. In particular, DARE messages MAY have multiple recipients 1236 and multiple signers. 1238 5.3.1. Structure: DareSigner 1240 The signature value 1242 Dig: String (Optional) Digest algorithm hint. Specifying the digest 1243 algorithm to be applied to the message body allows the body to be 1244 processed in streaming mode. 1246 Alg: String (Optional) Key exchange algorithm 1248 KeyIdentifier: String (Optional) Key identifier of the signature 1249 key. 1251 Certificate: X509Certificate (Optional) PKIX certificate of signer. 1253 Path: X509Certificate (Optional) PKIX certificates that establish a 1254 trust path for the signer. 1256 5.3.2. Structure: X509Certificate 1258 X5u: String (Optional) URL identifying an X.509 public key 1259 certificate 1261 X5: Binary (Optional) An X.509 public key certificate 1263 5.3.3. Structure: DareSignature 1265 Inherits: DareSigner 1267 The signature value 1269 Manifest: Binary (Optional) The data description that was signed. 1271 SignatureValue: Binary (Optional) The signature value as an Enhanced 1272 Data Sequence under the message Master Key. 1274 WitnessValue: Binary (Optional) The signature witness value used on 1275 an encrypted message to demonstrate that the signature was 1276 authorized by a party with actual knowledge of the encryption key 1277 used to encrypt the message. 1279 5.3.4. Structure: DareRecipient 1281 Recipient information 1283 KeyIdentifier: String (Optional) Key identifier for the encryption 1284 key. 1286 The Key identifier MUST be either a UDF fingerprint of a key or a 1287 Group Key Identifier 1289 KeyWrapDerivation: String (Optional) The key wrapping and derivation 1290 algorithms. 1292 WrappedMasterKey: Binary (Optional) The wrapped master key. The 1293 master key is encrypted under the result of the key exchange. 1295 RecipientKeyData: String (Optional) The per-recipient key exchange 1296 data. 1298 6. DARE Container Schema 1300 TBS stuff 1302 6.1. Container Headers 1304 TBS stuff 1306 6.1.1. Structure: ContainerEntry 1308 Inherits: ContainerHeader 1310 Inherits: ContainerHeader 1312 Body: Binary (Optional) The container data. 1314 6.1.2. Structure: ContainerHeaderFirst 1316 Inherits: ContainerHeader 1318 Inherits: ContainerHeader 1320 DataEncoding: String (Optional) Specifies the data encoding for the 1321 header section of for the following frames. This value is ONLY 1322 valid in Frame 0 which MUST have a header encoded in JSON. 1324 6.1.3. Structure: ContainerHeader 1326 Inherits: DareHeader 1328 Describes a container header. A container header MAY contain any 1329 DARE Message header. 1331 Index: Integer (Optional) The record index within the file. This 1332 MUST be unique and satisfy any additional requirements determined 1333 by the ContainerType. 1335 ContainerType: String (Optional) Specifies the container type for 1336 the following records. 1338 IsMeta: Boolean (Optional) If true, the current frame is a meta 1339 frame and does not contain a payload. 1341 Note: Meta frames MAY be present in any container. Applications 1342 MUST accept containers that contain meta frames at any position in 1343 the file. Applications MUST NOT interpret a meta frame as a data 1344 frame with an enpty payload. 1346 Default: Boolean (Optional) If set true in a persistent container, 1347 specifies that this record contains the default object for the 1348 container. 1350 ContentMeta: ContentMeta (Optional) Content meta data. 1352 TreePosition: Integer (Optional) Position of the frame containing 1353 the apex of the preceding sub-tree. 1355 IndexPosition: Integer (Optional) Specifies the position in the file 1356 at which the last index entry is to be found 1358 ExchangePosition: Integer (Optional) Specifies the position in the 1359 file at which the key exchange data is to be found 1361 ContainerIndex: ContainerIndex (Optional) An index of records in the 1362 current container up to but not including this one. 1364 First: Integer (Optional) Frame number of the first object instance 1365 value. 1367 Previous: Integer (Optional) Frame number of the immediately prior 1368 object instance value 1370 6.2. Content Metadata Structure 1372 TBS stuff 1374 6.2.1. Structure: ContentMeta 1376 Information describing the object instance 1378 ContentType: String (Optional) The content type field as specified 1379 in JWE 1381 Paths: String [0..Many] List of filename paths for the payload of 1382 the frame. 1384 UniqueID: String (Optional) Unique object identifier 1386 Created: DateTime (Optional) Initial creation date. 1388 Modified: DateTime (Optional) Date of last modification. 1390 6.3. Index Structures 1392 TBS stuff 1394 6.3.1. Structure: ContainerIndex 1396 A container index 1398 Full: Boolean (Optional) If true, the index is complete and contains 1399 position entries for all the frames in the file. If absent or 1400 false, the index is incremental and only contains position entries 1401 for records added since the last frame containing a 1402 ContainerIndex. 1404 Positions: IndexPosition [0..Many] List of container position 1405 entries 1407 Metas: IndexMeta [0..Many] List of container position entries 1409 6.3.2. Structure: IndexPosition 1411 Specifies the position in a file at which a specified record index is 1412 found 1414 Index: Integer (Optional) The record index within the file. 1416 Position: Integer (Optional) The record position within the file 1417 relative to the index base. 1419 6.3.3. Structure: KeyValue 1421 Specifies a key/value entry 1423 Key: String (Optional) The key 1425 Value: String (Optional) The value corresponding to the key 1427 6.3.4. Structure: IndexMeta 1429 Specifies the list of index entries at which a record with the 1430 specified metadata occurrs. 1432 Index: Integer [0..Many] List of record indicies within the file 1433 where frames matching the specified criteria are found. 1435 ContentType: String (Optional) Content type parameter 1437 Paths: String [0..Many] List of filename paths for the current 1438 frame. 1440 Labels: String [0..Many] List of labels that are applied to the 1441 current frame. 1443 7. Dare Container Applications 1445 DARE Containers are used to implement two forms of persistence store 1446 to support Mesh operations: 1448 Catalogs A set of related items which MAY be added, modified or 1449 deleted at any time. 1451 Spools A list of related items whose status MAY be changed at any 1452 time but which are immutable once added. 1454 Since DARE Containers are an append only log format, entries can only 1455 be modified or deleted by adding items to the log to change the 1456 status of previous entries. It is always possible to undo any 1457 operation on a catalog or spool unless the underlying container is 1458 purged or the individual entries modified. 1460 7.1. Catalog 1462 Catalogs contain a set of entries, each of which is distinguished by 1463 a unique identifier. 1465 Three operations are supported: 1467 Add Addition of the entry to the catalog 1469 Update Modification of the data associated with the entry excluding 1470 the identifier 1472 Delete Removal of the entry from the catalog 1474 The set of valid state transitions is defined by the Finite State 1475 machine: 1477 (Add-Update*-Delete)* 1479 Catalogs are used to represent sets of persistent objects associated 1480 with a Mesh Service Account. The user's set of contacts for example. 1481 Each contact entry may be modified many times over time but refers to 1482 the same subject for its entire lifetime. 1484 SchemaCatalog 1486 7.2. Spool 1488 Spools contain lists of entries, each of which is distinguished by a 1489 unique identifier. 1491 Four operations are supported: 1493 Post Addition of the entry to the spool 1495 Processed Marks the entry as having been processed. 1497 Unprocessed Returns the entry to the unread state. 1499 Delete Mark the entry as deleted allowing recovery of associated 1500 storage in a subsequent purge operation. 1502 The set of valid state transitions is defined by the Finite State 1503 machine: 1505 Post-(Processed| Unprocessed| Delete *) 1507 Spools are used to represent time sequence ordered entries such as 1508 lists of messages being sent or received, task queues and transaction 1509 logs. 1511 SchemaCatalog 1513 7.3. Archive 1515 A DARE Archive is a DARE Container whose entries contain files. This 1516 affords the same functionality as a traditional ZIP or tar archive 1517 but with the added cryptographic capabilities provided by the DARE 1518 format. 1520 8. Future Work 1522 The current specification describes an approach in which containers 1523 are written according to a strict append-only policy. Greater 1524 flexibility may be achieved by loosening this requirement allowing 1525 record(s) at the end of the container to be overwritten. 1527 8.1. Terminal integrity check 1529 A major concern when operating a critical service is the possibility 1530 of a hardware or power failure occurring during a write operation 1531 causing the file update to be incomplete. While most modern 1532 operating systems have effective mechanisms in place to prevent 1533 corruption of the file system itself in such circumstances, this does 1534 not provide sufficient protection at the application level. 1536 Appending a null record containing a container-specific magic number 1537 provides an effective means of detecting this circumstance that can 1538 be quickly verified. 1540 If a container specifies a terminal integrity check value in the 1541 header of frame zero, the container is considered to be in an 1542 incomplete write state if the final frame is not a null record 1543 specifying the magic number. 1545 When appending new records to such containers, the old terminal 1546 integrity check record is overwritten by the data being added and a 1547 new integrity check record appended to the end. 1549 8.2. Terminal index record 1551 A writer can maintain a complete (or partial) index of the container 1552 in its final record without additional space overhead by overwriting 1553 the prior index on each update. 1555 8.3. Deferred indexing 1557 The task of updating terminal indexes may be deferred to a time when 1558 the machine is not busy. This improves responsiveness and may avoid 1559 the need to re-index containers receiving a sequence of updates. 1561 This approach may be supported by appending new entries to the end of 1562 the container in the usual fashion and maintaining a record of 1563 containers to be updated as a separate task. 1565 When updating the index on a container that has been updated in this 1566 fashion, the writer must ensure that no data is lost even if the 1567 process is interrupted. The use of guard records and other 1568 precautions against loss of state is advised. 1570 9. Security Considerations 1572 This section describes security considerations arising from the use 1573 of DARE in general applications. 1575 Additional security considerations for use of DARE in Mesh services 1576 and applications are described in the Mesh Security Considerations 1577 guide [draft-hallambaker-mesh-security] . 1579 9.1. Encryption/Signature nesting 1581 9.2. Side channel 1583 9.3. Salt reuse 1585 10. IANA Considerations 1587 11. Acknowledgements 1589 A list of people who have contributed to the design of the Mesh is 1590 presented in [draft-hallambaker-mesh-architecture] . 1592 The name Data At Rest Encryption was proposed by Melhi Abdulhayo?lu. 1594 12. Appendix A: DARE Envelope Examples and Test Vectors 1596 13. Test Examples 1598 In the following examples, Alice's encryption private key parameters 1599 are: 1601 { 1602 "PrivateKeyECDH":{ 1603 "crv":"Ed25519", 1604 "Private":"0010U7hq2doyH5mcAW7IOwSWmtEBOoui0NPjMDWgzhE"}} 1606 Alice's signature private key parameters are: 1608 { 1609 "PrivateKeyECDH":{ 1610 "crv":"Ed25519", 1611 "Private":"bg3EBLIPaBnpxrf7EMpTMM205GahpJwwbr-QHhxdkHE"}} 1613 The body of the test message is the UTF8 representation of the 1614 following string: 1616 "This is a test long enough to require multiple blocks" 1618 The EDS sequences, are the UTF8 representation of the following 1619 strings: 1621 "Subject: Message metadata should be encrypted" 1622 "2018-02-01" 1624 13.1. Plaintext Message 1626 A plaintext message without associated EDS sequences is an empty 1627 header followed by the message body: 1629 { 1630 "DareEnvelope":[{}, 1631 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1632 BibG9ja3M" 1633 ]} 1635 13.2. Plaintext Message with EDS 1637 If a plaintext message contains EDS sequences, these are also in 1638 plaintext: 1640 { 1641 "DareEnvelope":[{ 1642 "Annotations":["iAEBiC1TdWJqZWN0OiBNZXNzYWdlIG1ldGFkYXRhIHNob3 1643 VsZCBiZSBlbmNyeXB0ZWSIAA", 1644 "iAECiAoyMDE4LTAyLTAxiAA" 1645 ]}, 1646 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1647 BibG9ja3M" 1648 ]} 1650 13.3. Encrypted Message 1652 The creator generates a master session key: 1654 E8 05 EC BE 68 65 64 5C A9 EE EF D7 6C 8A 1D 7F 1655 44 D5 06 7C 19 F4 4C 69 66 06 76 15 17 83 21 E0 1657 For each recipient of the message: 1659 The creator generates an ephemeral key: 1661 { 1662 "PrivateKeyECDH":{ 1663 "crv":"Ed25519", 1664 "Private":"w4MGsqG25drlx2c_-kFU0xj3FhOsRH1pZ5b7p_zwbuQ"}} 1666 The key agreement value is calculated: 1668 C9 D8 28 3B 0B E6 CD 89 EA 1A B3 27 9A 92 F2 BD 1669 90 34 A3 F8 2C 60 37 E1 94 91 3A A4 F1 92 2B 55 1671 The key agreement value is used as the input to a HKDF key derivation 1672 function with the info parameter master to create the key used to 1673 wrap the master key: 1675 D0 1A CD 52 28 CA 3F BB FB 1E 3B C7 7C C1 D7 0F 1676 AF B3 5C 3E 29 34 0E 10 DB E0 FC 07 71 CD 83 39 1678 The wrapped master key is: 1680 E0 99 45 98 B2 F7 DD B8 F3 C1 DF AC 96 D7 A4 66 1681 EB 73 DE BF 94 2E 85 4B 8D 0D 62 DF 63 B0 CE B3 1682 61 B7 06 1D 15 B4 CC CF 1684 This information is used to calculate the Recipient information shown 1685 in the example below. 1687 To encrypt a message, we first generate a unique salt value: 1689 7A FA 51 D6 D9 52 83 FD CD D8 40 77 C6 F9 27 43 1691 The salt value and master key are used to generate the payload 1692 encryption key: 1694 90 B0 FE BF 81 45 B1 07 2C 93 C5 5D 20 11 E4 C9 1695 8D B7 07 CB 14 1E A8 B8 1E 6B DA 77 76 D4 F5 71 1697 Since AES is a block cipher, we also require an initializarion 1698 vector: 1700 49 B5 07 6F 29 9D 6D 0E 5F 4C 41 58 3D 19 B4 35 1702 The output sequence is the encrypted bytes: 1704 D6 F8 01 E7 65 86 14 DB 37 91 48 60 5D 94 74 54 1705 BE 99 62 27 E9 0D BC 12 86 6A 80 DD 91 8F EC D8 1706 87 4B 63 22 B1 7F D2 1A A4 DE CD 79 06 1E 8A 75 1707 FD 7C 41 86 0D 72 38 49 F6 3E E5 18 4F B6 21 4A 1709 Since the message is not signed, there is no need for a trailer. The 1710 completed message is: 1712 { 1713 "DareEnvelope":[{ 1714 "enc":"A256CBC", 1715 "Salt":"evpR1tlSg_3N2EB3xvknQw", 1716 "recipients":[{ 1717 "kid":"MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 1718 "epk":{ 1719 "PublicKeyECDH":{ 1720 "crv":"Ed25519", 1721 "Public":"3slijATJracBxS1kJK9NkmM_OQt5AiVaKbUbhrDy2fg"}}, 1722 "wmk":"4JlFmLL33bjzwd-sltekZutz3r-ULoVLjQ1i32OwzrNhtwYdFb 1723 TMzw"} 1724 ]}, 1725 "1vgB52WGFNs3kUhgXZR0VL6ZYifpDbwShmqA3ZGP7NiHS2MisX_SGqTezXkGHo 1726 p1_XxBhg1yOEn2PuUYT7YhSg" 1727 ]} 1729 13.4. Signed Message 1731 Signed messages specify the digest algorithm to be used in the header 1732 and the signature value in the trailer. Note that the digest 1733 algorithm is not optional since it serves as notice that a decoder 1734 should digest the payload value to enable signature verification. 1736 { 1737 "DareEnvelope":[{ 1738 "dig":"S512"}, 1739 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBsZS 1740 BibG9ja3M", 1741 { 1742 "signatures":[{ 1743 "signature":"QDHtlnTnraUhIMSgsMwW8JRgE7o_HhDGq4aIPdIsrGml 1744 xCwcuiF827rAOURqmzr3075d8gcMpRtteL2uD6szAQ"} 1745 ], 1746 "PayloadDigest":"raim8SV5adPbWWn8FMM4mrRAQCO9A2jZ0NZAnFXWlG0x 1747 F6sWGJbnKSdtIJMmMU_hjarlIPEoY3vy9UdVlH5KAg"} 1748 ]} 1750 13.5. Signed and Encrypted Message 1752 A signed and encrypted message is encrypted and then signed. The 1753 signer proves knowledge of the payload plaintext by providing the 1754 plaintext witness value. 1756 { 1757 "DareEnvelope":[{ 1758 "enc":"A256CBC", 1759 "dig":"S512", 1760 "Salt":"2zyk-tgQTd-vxOSoOMOMNQ", 1761 "recipients":[{ 1762 "kid":"MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 1763 "epk":{ 1764 "PublicKeyECDH":{ 1765 "crv":"Ed25519", 1766 "Public":"muIeuoJKt-QBiPgmKYJkqvzlvybdidxnQ3EVfPLHOGc"}}, 1767 "wmk":"eTMPAU1wLBdh0dBzuZeF6nk-FdN6pTRtMREz3mXeEGGfjgszti 1768 PITA"} 1769 ]}, 1770 "9OfYXx5QpJVmbXAi0pgNkdaaR4OglRj35POxKjBN_aZzMF76TGcTjb7AbuKh55 1771 E0xQQNU10FgUksT7_5ScxQeQ", 1772 { 1773 "signatures":[{ 1774 "signature":"rSHVYYUXKXQY2zxQ6xGqUYJmcAgbkWZjMk3hMYtTukBp 1775 0mrVgaQSVzHOOWmTCi2Z8xcNG8vfXAV8faZr7BNNBA", 1776 "witness":"2j4zKpLY9PFBuIsnthlnKHZ4CIn9gZO9OrMwou6ZjRA"} 1777 ], 1778 "PayloadDigest":"fURpTxrZtuUDppoecrmr5xvSolZr2EsfZeIqhKDp7RII 1779 VxOlmSIri2JFgDxvVIXs49KbCTIS7hRn2_rxDUVtMw"} 1780 ]} 1782 14. Appendix B: DARE Container Examples and Test Vectors 1784 The data payloads in all the following examples are identical, only 1785 the authentication and/or encryption is different. 1787 o Frame 1..n consists of 300 bytes being the byte sequence 00, 01, 1788 02, etc. repeating after 256 bytes. 1790 For conciseness, the raw data format is omitted for examples after 1791 the first, except where the data payload has been transformed, (i.e. 1792 encrypted). 1794 14.1. Simple container 1796 the following example shows a simple container with first frame and a 1797 single data frame: 1799 f4 5d 1800 f0 59 1801 f0 00 1802 5d f4 1803 f5 01 40 1804 f0 0f 1805 f1 01 2c 1806 40 01 f5 1808 Since there is no integrity check, there is no need for trailer 1809 entries. The header values are: 1811 Frame 0 1813 { 1814 "Index": 0, 1815 "ContainerType": "List", 1816 "ContentMeta": {}, 1817 "DataEncoding": "JSON"} 1819 [Empty trailer] 1821 Frame 1 1823 { 1824 "Index": 1} 1826 [Empty trailer] 1828 14.2. Payload and chain digests 1830 The following example shows a chain container with a first frame and 1831 three data frames. The headers of these frames is the same as before 1832 but the frames now have trailers specifying the PayloadDigest and 1833 ChainDigest values: 1835 Frame 0 1837 { 1838 "Index": 0, 1839 "ContainerType": "Chain", 1840 "ContentMeta": {}, 1841 "DataEncoding": "JSON"} 1843 [Empty trailer] 1845 Frame 1 1847 { 1848 "Index": 1} 1850 { 1851 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1852 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1853 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1854 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1856 Frame 2 1858 { 1859 "Index": 2} 1861 { 1862 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1863 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1864 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1865 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1867 Frame 3 1868 { 1869 "Index": 3} 1871 { 1872 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1873 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1874 "ChainDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1875 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1877 14.3. Merkle Tree 1879 The following example shows a chain container with a first frame and 1880 six data frames. The trailers now contain the TreePosition and 1881 TreeDigest values: 1883 Frame 0 1885 { 1886 "Index": 0, 1887 "ContainerType": "Merkle", 1888 "ContentMeta": {}, 1889 "DataEncoding": "JSON"} 1891 [Empty trailer] 1893 Frame 1 1895 { 1896 "Index": 1, 1897 "TreePosition": 0} 1899 { 1900 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1901 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1902 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1903 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1905 Frame 2 1907 { 1908 "Index": 2, 1909 "TreePosition": 325} 1911 { 1912 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1913 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1914 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 1915 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 1917 Frame 3 1919 { 1920 "Index": 3, 1921 "TreePosition": 325} 1923 { 1924 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1925 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1926 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1927 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1929 Frame 4 1931 { 1932 "Index": 4, 1933 "TreePosition": 1469} 1935 { 1936 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1937 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1938 "TreeDigest": "vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAvK 1939 Sj8gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"} 1941 Frame 5 1943 { 1944 "Index": 5, 1945 "TreePosition": 1469} 1947 { 1948 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1949 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1950 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1951 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1953 Frame 6 1955 { 1956 "Index": 6, 1957 "TreePosition": 2616} 1959 { 1960 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1961 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1962 "TreeDigest": "WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9S 1963 Z0mdr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"} 1965 14.4. Signed container 1967 The following example shows a tree container with a signature in the 1968 final record. The signing key parameters are: 1970 { 1971 "PrivateKeyECDH":{ 1972 "crv":"Ed25519", 1973 "Private":"bg3EBLIPaBnpxrf7EMpTMM205GahpJwwbr-QHhxdkHE"}} 1975 The container headers and trailers are: 1977 Frame 0 1979 { 1980 "Index": 0, 1981 "ContainerType": "Merkle", 1982 "ContentMeta": {}, 1983 "DataEncoding": "JSON"} 1985 [Empty trailer] 1987 Frame 1 1989 { 1990 "Index": 1, 1991 "TreePosition": 0} 1993 { 1994 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 1995 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1996 "TreeDigest": "T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVRV 1997 z9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"} 1999 Frame 2 2001 { 2002 "Index": 2, 2003 "TreePosition": 325} 2005 { 2006 "PayloadDigest": "8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZD 2007 lZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2008 "TreeDigest": "7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kokk 2009 FnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"} 2011 14.5. Encrypted container 2013 The following example shows a container in which all the frame 2014 payloads are encrypted under the same master secret established in a 2015 key agreement specified in the first frame. 2017 Frame 0 2019 { 2020 "enc": "A256CBC", 2021 "Salt": "VUkvKcUljfhCXRdwOIUKXw", 2022 "recipients": [{ 2023 "kid": "MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 2024 "epk": { 2025 "PublicKeyECDH": { 2026 "crv": "Ed25519", 2027 "Public": "uQ_tso7yyIfT_iWkKF99RdyT2nr09AMFsBunz_Nn1Rs"}}, 2028 "wmk": "Vf7Bm8mlfaCVHCw2WQKVap6gqEQNUs8a6crfVUSocZf1p3h8fwQXag"}], 2029 "Index": 0, 2030 "ContainerType": "List", 2031 "ContentMeta": {}, 2032 "DataEncoding": "JSON"} 2034 [Empty trailer] 2036 Frame 1 2038 { 2039 "enc": "A256CBC", 2040 "Salt": "TiYLR8rcEcQ5PBo9sfFxOA", 2041 "Index": 1} 2043 [Empty trailer] 2045 Frame 2 2047 { 2048 "enc": "A256CBC", 2049 "Salt": "6uLkyxmQXH9RpSvzSoLvcw", 2050 "Index": 2} 2052 [Empty trailer] 2054 Here are the container bytes. Note that the content is now encrypted 2055 and has expanded by 25 bytes. These are the salt (16 bytes), the AES 2056 padding (4 bytes) and the JSON-B framing (5 bytes). 2058 f5 01 c0 2059 f1 01 ab 2060 f0 10 2061 c0 01 f5 2062 f5 01 7c 2063 f0 47 2064 f1 01 30 2065 7c 01 f5 2066 f5 01 7c 2067 f0 47 2068 f1 01 30 2069 7c 01 f5 2071 The following example shows a container in which all the frame 2072 payloads are encrypted under separate key agreements specified in the 2073 payload frames. 2075 Frame 0 2077 { 2078 "Index": 0, 2079 "ContainerType": "List", 2080 "ContentMeta": {}, 2081 "DataEncoding": "JSON"} 2083 [Empty trailer] 2085 Frame 1 2087 { 2088 "enc": "A256CBC", 2089 "Salt": "e0YcTvqJes01YzetVlqGHw", 2090 "recipients": [{ 2091 "kid": "MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 2092 "epk": { 2093 "PublicKeyECDH": { 2094 "crv": "Ed25519", 2095 "Public": "nSY_Igk_Z206hWsJlRDoqqVvPuzlQgErXoysr-PQAFg"}}, 2096 "wmk": "ph6_MrXLOAikbkk7yaIQA-tmqe2yecNC0b0P_1_ANpVo06bFDlQr6Q"}], 2097 "Index": 1} 2099 [Empty trailer] 2101 Frame 2 2103 { 2104 "enc": "A256CBC", 2105 "Salt": "jtPTRwi1Kgfem7JHLavl2A", 2106 "recipients": [{ 2107 "kid": "MBNC-3DR6-VILB-CGQL-BEB7-55EU-EZZY", 2108 "epk": { 2109 "PublicKeyECDH": { 2110 "crv": "Ed25519", 2111 "Public": "FqsZguMDMTkqTtIgQ5gPJwHyltHKZUl4Z4BTBfSiUt4"}}, 2112 "wmk": "JY_GaZkr7LdSCuYwYH7zB0tesLouy2wu7tBwwUChnqfZ828XUELUYg"}], 2113 "Index": 2} 2115 [Empty trailer] 2117 15. Appendix C: Previous Frame Function 2119 public long PreviousFrame (long Frame) { 2120 long x2 = Frame + 1; 2121 long d = 1; 2123 while (x2 > 0) { 2124 if ((x2 & 1) == 1) { 2125 return x2 == 1 ? (d / 2) - 1 : Frame - d; 2126 } 2127 d = d * 2; 2128 x2 = x2 / 2; 2129 } 2130 return 0; 2131 } 2133 16. Appendix D: Outstanding Issues 2135 The following issues need to be addressed. 2137 +----------------+--------------------------------------------------+ 2138 | Issue | Description | 2139 +----------------+--------------------------------------------------+ 2140 | X25519 | The examples currently use Edwards Curve25519 | 2141 | | for encryption. This should be Curve X25519 | 2142 | Indexing | No examples are given of indexing a container | 2143 | Archive | Should include a file archive example | 2144 | File Path | Mention the file path security issue in the | 2145 | | security considerations | 2146 | Security | Write Security considerations | 2147 | Considerations | | 2148 | AES-GCM | Switch to using AES GCM in the examples | 2149 | Witness | Complete handling of witness values. | 2150 | Schema | Complete the schema documentation | 2151 | Container Redo | Rework the container/header objects so that | 2152 | | these are separate classes and Header is an | 2153 | | entry in the Container header. | 2154 +----------------+--------------------------------------------------+ 2156 Table 1 2158 17. References 2160 17.1. Normative References 2162 [draft-hallambaker-jsonbcd] 2163 Hallam-Baker, P., "Binary Encodings for JavaScript Object 2164 Notation: JSON-B, JSON-C, JSON-D", draft-hallambaker- 2165 jsonbcd-14 (work in progress), April 2019. 2167 [draft-hallambaker-mesh-architecture] 2168 Hallam-Baker, P., "Mathematical Mesh 3.0 Part I: 2169 Architecture Guide", draft-hallambaker-mesh- 2170 architecture-08 (work in progress), July 2019. 2172 [draft-hallambaker-mesh-security] 2173 Hallam-Baker, P., "Mathematical Mesh Part VII: Security 2174 Considerations", draft-hallambaker-mesh-security-00 (work 2175 in progress), April 2019. 2177 [draft-hallambaker-mesh-udf] 2178 Hallam-Baker, P., "Mathematical Mesh 3.0 Part II: Uniform 2179 Data Fingerprint.", draft-hallambaker-mesh-udf-03 (work in 2180 progress), July 2019. 2182 [IANAJOSE] 2183 "[Reference Not Found!]". 2185 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2186 Requirement Levels", BCP 14, RFC 2119, 2187 DOI 10.17487/RFC2119, March 1997. 2189 [RFC2315] Kaliski, B., "PKCS #7: Cryptographic Message Syntax 2190 Version 1.5", RFC 2315, DOI 10.17487/RFC2315, March 1998. 2192 [RFC3394] Schaad, J. and R. Housley, "Advanced Encryption Standard 2193 (AES) Key Wrap Algorithm", RFC 3394, DOI 10.17487/RFC3394, 2194 September 2002. 2196 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 2197 Thayer, "OpenPGP Message Format", RFC 4880, 2198 DOI 10.17487/RFC4880, November 2007. 2200 [RFC4949] Shirey, R., "Internet Security Glossary, Version 2", 2201 FYI 36, RFC 4949, DOI 10.17487/RFC4949, August 2007. 2203 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2204 Key Derivation Function (HKDF)", RFC 5869, 2205 DOI 10.17487/RFC5869, May 2010. 2207 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2208 Specifications and Registration Procedures", BCP 13, 2209 RFC 6838, DOI 10.17487/RFC6838, January 2013. 2211 [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data 2212 Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2213 2014. 2215 [RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web 2216 Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2217 2015. 2219 [RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", 2220 RFC 7516, DOI 10.17487/RFC7516, May 2015. 2222 [RFC7517] Jones, M., "JSON Web Key (JWK)", RFC 7517, 2223 DOI 10.17487/RFC7517, May 2015. 2225 [RFC7518] Jones, M., "JSON Web Algorithms (JWA)", RFC 7518, 2226 DOI 10.17487/RFC7518, May 2015. 2228 17.2. Informative References 2230 [BLOCKCHAIN] 2231 Chain.com, "Blockchain Specification". 2233 [Davis2001] 2234 Davis, D., "Defective Sign & Encrypt in S/MIME, PKCS#7, 2235 MOSS, PEM, PGP, and XML", May 2001. 2237 [RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, 2238 RFC 5652, DOI 10.17487/RFC5652, September 2009. 2240 [ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format 2241 Specification", October 2014. 2243 17.3. URIs 2245 [1] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2247 [2] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2249 [3] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2251 [4] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2253 [5] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2255 [6] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2257 [7] http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html 2259 Author's Address 2261 Phillip Hallam-Baker 2263 Email: phill@hallambaker.com