idnits 2.17.1 draft-hallambaker-mesh-dare-15.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Authors' Addresses Section. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (20 April 2022) is 737 days in the past. Is this intentional? Checking references for intended status: Informational ---------------------------------------------------------------------------- == Missing Reference: 'EOF' is mentioned on line 718, but not defined ** Obsolete normative reference: RFC 7159 (Obsoleted by RFC 8259) Summary: 2 errors (**), 0 flaws (~~), 2 warnings (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group P. M. Hallam-Baker 3 Internet-Draft ThresholdSecrets.com 4 Intended status: Informational 20 April 2022 5 Expires: 22 October 2022 7 Mathematical Mesh 3.0 Part III : Data At Rest Encryption (DARE) 8 draft-hallambaker-mesh-dare-15 10 Abstract 12 This document describes the Data At Rest Encryption (DARE) Envelope 13 and Sequence syntax. 15 The DARE Envelope syntax is used to digitally sign, digest, 16 authenticate, or encrypt arbitrary content data. 18 The DARE Sequence syntax describes an append-only sequence of 19 entries, each containing a DARE Envelope. DARE Sequences may support 20 cryptographic integrity verification of the entire data container 21 content by means of a Merkle tree. 23 [Note to Readers] 25 Discussion of this draft takes place on the MATHMESH mailing list 26 (mathmesh@ietf.org), which is archived at 27 https://mailarchive.ietf.org/arch/search/?email_list=mathmesh. 29 This document is also available online at 30 http://mathmesh.com/Documents/draft-hallambaker-mesh-dare.html. 32 Status of This Memo 34 This Internet-Draft is submitted in full conformance with the 35 provisions of BCP 78 and BCP 79. 37 Internet-Drafts are working documents of the Internet Engineering 38 Task Force (IETF). Note that other groups may also distribute 39 working documents as Internet-Drafts. The list of current Internet- 40 Drafts is at https://datatracker.ietf.org/drafts/current/. 42 Internet-Drafts are draft documents valid for a maximum of six months 43 and may be updated, replaced, or obsoleted by other documents at any 44 time. It is inappropriate to use Internet-Drafts as reference 45 material or to cite them other than as "work in progress." 47 This Internet-Draft will expire on 22 October 2022. 49 Copyright Notice 51 Copyright (c) 2022 IETF Trust and the persons identified as the 52 document authors. All rights reserved. 54 This document is subject to BCP 78 and the IETF Trust's Legal 55 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 56 license-info) in effect on the date of publication of this document. 57 Please review these documents carefully, as they describe your rights 58 and restrictions with respect to this document. 60 Table of Contents 62 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 63 1.1. Encryption and Integrity . . . . . . . . . . . . . . . . 5 64 1.1.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 5 65 1.1.2. Data Erasure . . . . . . . . . . . . . . . . . . . . 6 66 1.2. Signature . . . . . . . . . . . . . . . . . . . . . . . . 6 67 1.2.1. Signing Individual Plaintext Envelopes . . . . . . . 7 68 1.2.2. Signing Individual Encrypted Envelopes . . . . . . . 7 69 1.2.3. Signing sequences of envelopes . . . . . . . . . . . 7 70 1.3. Sequence . . . . . . . . . . . . . . . . . . . . . . . . 8 71 1.3.1. Sequence Format . . . . . . . . . . . . . . . . . . . 8 72 1.3.2. Write . . . . . . . . . . . . . . . . . . . . . . . . 9 73 1.3.3. Encryption and Authentication . . . . . . . . . . . . 9 74 1.3.4. Integrity and Signature . . . . . . . . . . . . . . . 10 75 1.3.5. Redaction . . . . . . . . . . . . . . . . . . . . . . 10 76 1.3.6. Alternative approaches . . . . . . . . . . . . . . . 11 77 1.3.7. Efficiency . . . . . . . . . . . . . . . . . . . . . 11 78 2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 12 79 2.1. Related Specifications . . . . . . . . . . . . . . . . . 12 80 2.2. Requirements Language . . . . . . . . . . . . . . . . . . 13 81 2.3. Defined terms . . . . . . . . . . . . . . . . . . . . . . 13 82 3. DARE Envelope Architecture . . . . . . . . . . . . . . . . . 14 83 3.1. Processing Considerations . . . . . . . . . . . . . . . . 15 84 3.2. Encoded Data Sequence . . . . . . . . . . . . . . . . . . 15 85 3.3. Content Metadata and Annotations . . . . . . . . . . . . 16 86 3.4. Encryption and Integrity . . . . . . . . . . . . . . . . 17 87 3.4.1. Key Exchange . . . . . . . . . . . . . . . . . . . . 18 88 3.4.2. Key Identifiers . . . . . . . . . . . . . . . . . . . 19 89 3.4.3. Salt Derivation . . . . . . . . . . . . . . . . . . . 19 90 3.4.4. Key Derivation . . . . . . . . . . . . . . . . . . . 20 91 3.5. Signature . . . . . . . . . . . . . . . . . . . . . . . . 21 92 3.6. Algorithms . . . . . . . . . . . . . . . . . . . . . . . 21 93 3.6.1. Field: kwd . . . . . . . . . . . . . . . . . . . . . 21 94 4. DARE Sequence Architecture . . . . . . . . . . . . . . . . . 21 95 4.1. Sequence Navigation . . . . . . . . . . . . . . . . . . . 21 96 4.1.1. Tree . . . . . . . . . . . . . . . . . . . . . . . . 22 97 4.1.2. Position Index . . . . . . . . . . . . . . . . . . . 23 98 4.1.3. Metadata Index . . . . . . . . . . . . . . . . . . . 23 99 4.2. Integrity Mechanisms . . . . . . . . . . . . . . . . . . 23 100 4.2.1. Digest Chain calculation . . . . . . . . . . . . . . 23 101 4.2.2. Binary Merkle tree calculation . . . . . . . . . . . 24 102 4.2.3. Signature . . . . . . . . . . . . . . . . . . . . . . 24 103 5. DARE Schema . . . . . . . . . . . . . . . . . . . . . . . . . 25 104 5.1. Envelope Classes . . . . . . . . . . . . . . . . . . . . 25 105 5.1.1. Structure: DareEnvelopeSequence . . . . . . . . . . . 25 106 5.2. Header and Trailer Classes . . . . . . . . . . . . . . . 25 107 5.2.1. Structure: DareTrailer . . . . . . . . . . . . . . . 25 108 5.2.2. Structure: DareHeader . . . . . . . . . . . . . . . . 26 109 5.2.3. Structure: ContentMeta . . . . . . . . . . . . . . . 27 110 5.3. Cryptographic Data . . . . . . . . . . . . . . . . . . . 28 111 5.3.1. Structure: DareSignature . . . . . . . . . . . . . . 28 112 5.3.2. Structure: X509Certificate . . . . . . . . . . . . . 29 113 5.3.3. Structure: DareRecipient . . . . . . . . . . . . . . 29 114 5.3.4. Structure: DarePolicy . . . . . . . . . . . . . . . . 29 115 5.3.5. Structure: FileEntry . . . . . . . . . . . . . . . . 30 116 6. DARE Container Schema . . . . . . . . . . . . . . . . . . . . 30 117 6.1. Container Headers . . . . . . . . . . . . . . . . . . . . 30 118 6.1.1. Structure: SequenceInfo . . . . . . . . . . . . . . . 30 119 6.2. Index Structures . . . . . . . . . . . . . . . . . . . . 31 120 6.2.1. Structure: SequenceIndex . . . . . . . . . . . . . . 31 121 6.2.2. Structure: IndexPosition . . . . . . . . . . . . . . 32 122 6.2.3. Structure: KeyValue . . . . . . . . . . . . . . . . . 32 123 7. Dare Sequence Applications . . . . . . . . . . . . . . . . . 32 124 7.1. Catalog . . . . . . . . . . . . . . . . . . . . . . . . . 32 125 7.2. Spool . . . . . . . . . . . . . . . . . . . . . . . . . . 33 126 7.3. Archive . . . . . . . . . . . . . . . . . . . . . . . . . 33 127 8. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . 34 128 8.1. Terminal integrity check . . . . . . . . . . . . . . . . 34 129 8.2. Terminal index record . . . . . . . . . . . . . . . . . . 34 130 8.3. Deferred indexing . . . . . . . . . . . . . . . . . . . . 34 131 9. Security Considerations . . . . . . . . . . . . . . . . . . . 35 132 9.1. Encryption/Signature nesting . . . . . . . . . . . . . . 35 133 9.2. Side channel . . . . . . . . . . . . . . . . . . . . . . 35 134 9.3. Salt reuse . . . . . . . . . . . . . . . . . . . . . . . 35 135 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35 136 11. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 35 137 12. Appendix A: DARE Envelope Examples and Test Vectors . . . . . 35 138 13. Test Examples . . . . . . . . . . . . . . . . . . . . . . . . 35 139 13.1. Plaintext Message . . . . . . . . . . . . . . . . . . . 36 140 13.2. Plaintext Message with EDS . . . . . . . . . . . . . . . 36 141 13.3. Encrypted Message . . . . . . . . . . . . . . . . . . . 36 142 13.4. Signed Message . . . . . . . . . . . . . . . . . . . . . 38 143 13.5. Signed and Encrypted Message . . . . . . . . . . . . . . 39 144 14. Appendix B: DARE Sequence Examples and Test Vectors . . . . . 39 145 14.1. Simple sequence . . . . . . . . . . . . . . . . . . . . 40 146 14.2. Payload and chain digests . . . . . . . . . . . . . . . 40 147 14.3. Merkle Tree . . . . . . . . . . . . . . . . . . . . . . 42 148 14.4. Signed sequence . . . . . . . . . . . . . . . . . . . . 45 149 14.5. Encrypted sequence . . . . . . . . . . . . . . . . . . . 46 150 15. Appendix C: Previous Frame Function . . . . . . . . . . . . . 50 151 16. Appendix D: Outstanding Issues . . . . . . . . . . . . . . . 50 152 17. Normative References . . . . . . . . . . . . . . . . . . . . 51 153 18. Informative References . . . . . . . . . . . . . . . . . . . 53 155 1. Introduction 157 This document describes the Data At Rest Encryption (DARE) Envelope 158 and Sequence Syntax. The DARE Envelope syntax is used to digitally 159 sign, digest, authenticate, or encrypt arbitrary message content. 160 The DARE Sequence syntax describes an append-only sequence of data 161 frames, each containing a DARE Envelope that supports efficient 162 incremental signature and encryption. 164 The DARE Envelope Syntax is based on a subset of the JSON Web 165 Signature [RFC7515] and JSON Web Encryption [RFC7516] standards and 166 shares many fields and semantics. The processing model and data 167 structures have been streamlined to remove alternative means of 168 specifying the same content and to enable multiple data sequences to 169 be signed and encrypted under a single master encryption key without 170 compromise to security. 172 A DARE Envelope consists of a _Header_, _Payload_ and an optional 173 _Trailer_. To enable single pass encoding and decoding, the Header 174 contains all the information required to perform cryptographic 175 processing of the Payload and authentication data (digest, MAC, 176 signature values) MAY be deferred to the Trailer section. 178 A DARE Sequence is an append-only log format consisting of a sequence 179 of frames. Cryptographic enhancements (signature, encryption) may be 180 applied to individual frames or to sets of frames. Thus, a single 181 key exchange may be used to provide a master key to encrypt multiple 182 frames and a single signature may be used to authenticate all the 183 frames in the container up to and including the frame in which the 184 signature is presented. 186 The DARE Envelope syntax may be used either as a standalone 187 cryptographic message syntax or as a means of presenting a single 188 DARE Sequence frame together with the complete cryptographic context 189 required to verify the contents and decrypt them. 191 1.1. Encryption and Integrity 193 A key innovation in the DARE Envelope Syntax is the separation of key 194 exchange and data encryption operations so that a Master Key (MK) 195 established in a single exchange to be applied to multiple data 196 sequences. This means that a single public key operation MAY be used 197 to encrypt and/or authenticate multiple parts of the same DARE 198 Envelope or multiple frames in a DARE Sequence. 200 To avoid reuse of the key and to avoid the need to communicate 201 separate IVs, each octet sequence is encrypted under a different 202 encryption key (and IV if required) derived from the Master Key by 203 means of a salt that is unique for each octet sequence that is 204 encrypted. The same approach is used to generate keys for 205 calculating a MAC over the octet sequence if required. This approach 206 allows encryption and integrity protections to be applied to the 207 envelope payload, to header or trailer fields or to application 208 defined Enhanced Data Sequences in the header or trailer. 210 1.1.1. Key Exchange 212 Traditional cryptographic containers describe the application of a 213 single key exchange to encryption of a single octet sequence. 214 Examples include PCKS#7/CMS [RFC2315], OpenPGP [RFC4880] and JSON Web 215 Encryption [RFC7516]. 217 To encrypt data using RSA, the encoder first generates a random 218 encryption key and initialization vector (IV). The encryption key is 219 encrypted under the public key of each recipient to create a per- 220 recipient decryption entry. The encryption key, plaintext and IV are 221 used to generate the ciphertext (figure 1). 223 (Artwork only available as svg: No external link available, see 224 draft-hallambaker-mesh-dare-15.html for artwork.) 226 Figure 1: Monolithic Key Exchange and Encrypt 228 This approach is adequate for the task of encrypting a single octet 229 stream. It is less than satisfactory when encrypting multiple octet 230 streams or very long streams for which a rekeying operation is 231 desirable. 233 In the DARE approach, key exchange and key derivation are separate 234 operations and keys MAY be derived for encryption or integrity 235 purposes or both. A single key exchange MAY be used to derive keys 236 to apply encryption and integrity enhancements to multiple data 237 sequences. 239 The DARE key exchange begins with the same key exchange used to 240 produce the CEK in JWE but instead of using the CEK to encipher data 241 directly, it is used as one of the inputs to a Key Derivation 242 Function (KDF) that is used to derive parameters for each block of 243 data to be encrypted. To avoid the need to introduce additional 244 terminology, the term 'CEK' is still used to describe the output of 245 the key agreement algorithm (including key unwrapping if required) 246 but it is more appropriately described as a Master Key (figure 2). 248 (Artwork only available as svg: No external link available, see 249 draft-hallambaker-mesh-dare-15.html for artwork.) 251 Figure 2: Exchange of Master Key 253 A Master Key may be used to encrypt any number of data items. Each 254 data item is encrypted under a different encryption key and IV (if 255 required). This data is derived from the Master Key using the HKDF 256 function [RFC5869] using a different salt for each data item and 257 separate info tags for each cryptographic function (figure 3). 259 (Artwork only available as svg: No external link available, see 260 draft-hallambaker-mesh-dare-15.html for artwork.) 262 Figure 3: Data item encryption under Master Key and per-item salt. 264 This approach to encryption offers considerably greater flexibility 265 allowing the same format for data item encryption to be applied at 266 the transport, message or field level. 268 1.1.2. Data Erasure 270 Each encrypted DARE Envelope specifies a unique Master Salt value of 271 at least 128 bits which is used to derive the salt values used to 272 derive cryptographic keys for the envelope payload and annotations. 274 Erasure of the Master Salt value MAY be used to effectively render 275 the envelope payload and annotations undecipherable without altering 276 the envelope payload data. The work factor for decryption will be 277 O(2^128) even if the decryption key is compromised. 279 1.2. Signature 281 As with encryption, DARE Envelope signatures MAY be applied to an 282 individual envelope or a sequence of envelope. 284 1.2.1. Signing Individual Plaintext Envelopes 286 When an individual plaintext envelope is signed, the digest value 287 used to create the signature is calculated over the binary value of 288 the payload data. That is, the value of the payload before the 289 encoding (Base-64, JSON-B) is applied. 291 1.2.2. Signing Individual Encrypted Envelopes 293 When an individual plaintext envelope is signed, the digest value 294 used to create the signature is calculated over the binary value of 295 the payload data. That is, the value of the payload after encryption 296 but before the encoding (Base-64, JSON-B) is applied. 298 Use of signing and encryption in combination presents the risk of 299 subtle attacks depending on the order in which signing and encryption 300 take place [Davis2001]. 302 Na?ve approaches in which an envelope is encrypted and then signed 303 present the possibility of a surreptitious forwarding attack. For 304 example, Alice signs an envelope and sends it to Mallet who then 305 strips off Alice's signature and sends the envelope to Bob. 307 Na?ve approaches in which an envelope is signed and then encrypted 308 present the possibility of an attacker claiming authorship of a 309 ciphertext. For example, Alice encrypts a ciphertext for Bob and 310 then signs it. Mallet then intercepts the envelope and sends it to 311 Bob. 313 While neither attack is a concern in all applications, both attacks 314 pose potential hazards for the unwary and require close inspection of 315 application protocol design to avoid exploitation. 317 To prevent these attacks, each signature on an envelope that is 318 signed and encrypted MUST include a witness value that is calculated 319 by applying a MAC function to the signature value as described in 320 section XXX. 322 1.2.3. Signing sequences of envelopes 324 To sign multiple envelopes with a single signature, we first 325 construct a Merkle tree of the envelope payload digest values and 326 then sign the root of the Merkle tree. 328 [This is not yet implemented but will be soon.] 330 1.3. Sequence 332 DARE Sequence is a message and file syntax that allows a sequence of 333 data frames to be represented with cryptographic integrity, 334 signature, and encryption enhancements to be constructed in an append 335 only format. 337 The format is designed to meet the requirements of a wide range of 338 use cases including: 340 * Recording transactions in persistent storage. 342 * Synchronizing transaction logs between hosts. 344 * File archive. 346 * Message spool. 348 * Signing and encrypting single data items. 350 * Incremental encryption and authentication of server logs. 352 1.3.1. Sequence Format 354 A Sequence consists of a sequence of variable length Frames. Each 355 frame consists of a forward length indicator, the framed data and a 356 reverse length indicator. The reverse length indicator is written 357 out backwards allowing the length and thus the frame to be read in 358 the reverse direction: 360 (Artwork only available as svg: No external link available, see 361 draft-hallambaker-mesh-dare-15.html for artwork.) 363 Figure 4: JBCD Bidirectional Frame 365 Each frame contains a single DARE Envelope consisting of a Header, 366 Payload and Trailer (if required). The first frame in a container 367 describes the container format options and defaults. These include 368 the range of encoding options for frame metadata supported and the 369 container profiles to which the container conforms. 371 All internal data formats support use of pointers of up to 64 bits 372 allowing containers of up to 18 exabytes to be written. 374 Five container types are currently specified: 376 Simple The container does not provide any index or content integrity 377 checks. 379 Tree Frame headers contain entries that specify the start position 380 of previous frames at the apex of the immediately enclosing binary 381 tree. This enables efficient random access to any frame in the 382 file. 384 Digest Each frame trailer contains a PayloadDigest field. 385 Modification of the payload will cause verification of the 386 PayloadDigest value to fail on that frame. 388 Chain Each frame trailer contains PayloadDigest and ChainDigest 389 fields allowing modifications to the payload data to be detected. 390 Modification of the payload will cause verification of the 391 PayloadDigest value to fail on that frame and verification of the 392 ChainDigest value to fail on all subsequent frames. 394 Merkle Tree Frame headers contain entries that specify the start 395 position of previous frames at the apex of the immediately 396 enclosing binary tree. Frame Trailers contain TreeDigestPartial 397 and TreeDigestFinal entries forming a Merkle digest tree. 399 Currently, the Mesh only makes use of the Merkle Tree sequence type. 401 1.3.2. Write 403 In normal circumstances, Sequences are written as an append only log. 404 As with Envelopes, integrity information (payload digest, signatures) 405 is written to the entry trailer. Thus, large payloads may be written 406 without the need to buffer the payload data _provided that_ the 407 content length is known in advance. 409 Should exceptional circumstances require, Sequence entries MAY be 410 erased by overwriting the Payload and/or parts of the Header content 411 without compromising the ability to verify other entries in the 412 container. If the entry Payload is encrypted, it is sufficient to 413 erase the container salt value to render the container entry 414 effectively inaccessible (though recovery might still be possible if 415 the original salt value can be recovered from the storage media. 417 1.3.3. Encryption and Authentication 419 Frame payloads and associated attributes MAY be encrypted and/or 420 authenticated in the same manner as Envelopes. 422 _Incremental encryption_ is supported allowing encryption parameters 423 from a single public key exchange operation to be applied to encrypt 424 multiple frames. The public key exchange information is specified in 425 the first encrypted frame and subsequent frames encrypted under those 426 parameters specify the location at which the key exchange information 427 is to be found by means of the ExchangePosition field which MUST 428 specify a location that is earlier in the file. 430 To avoid cryptographic vulnerabilities resulting from key re-use, the 431 DARE key exchange requires that each encrypted sequence use an 432 encryption key and initialization vector derived from the master key 433 established in the public key exchange by means of a unique salt 434 specified in each envelope. 436 Each Envelope and by extension, each Sequence frame MUST specify a 437 unique salt value of at least 128 bits. Since the encryption key is 438 derived from the salt value by means of a Key Derivation Function, 439 erasure of the salt MAY be used as a means of rendering the payload 440 plaintext value inaccessible without changing the payload value. 442 1.3.4. Integrity and Signature 444 Signatures MAY be applied to a payload digest, the final digest in a 445 chain or tree. The chain and tree digest modes allow a single 446 signature to be used to authenticate all frame payloads in a 447 container. 449 The tree signature mode is particularly suited to applications such 450 as file archives as it allows files to be verified individually 451 without requiring the signer to sign each individually. Furthermore, 452 in applications such as code signing, it allows a single signature to 453 be used to verify both the integrity of the code and its membership 454 of the distribution. 456 As with DARE Envelope, the signature mechanism does not specify the 457 interpretation of the signature semantics. The presence of a 458 signature demonstrates that the holder of the private key applied it 459 to the specified digest value but not their motive for doing so. 460 Describing such semantics is beyond the scope of this document and is 461 deferred to future work. 463 1.3.5. Redaction 465 The chief disadvantage of using an append-only format is that 466 containers only increase in size. In many applications, much of the 467 data in the container becomes redundant or obsolete and a process 468 analogous to garbage collection is required. This process is called 469 _redaction_. 471 The simplest method of redaction is to create a new container and 472 sequentially copy each entry from the old container to the new, 473 discarding redundant frames and obsolete header information. 475 For example, partial index records may be consolidated into a single 476 index record placed in the last frame of the container. Unnecessary 477 signature and integrity data may be discarded and so on. 479 While redaction could in principle be effected by moving data in- 480 place in the existing container, supporting this approach in a robust 481 fashion is considerably more complex as it requires backward 482 references in subsequent frames to be overridden as each frame is 483 moved. 485 1.3.6. Alternative approaches 487 Many file proprietary formats are in use that support some or all of 488 these capabilities but only a handful have public, let alone open, 489 standards. DARE Sequence is designed to provide a superset of the 490 capabilities of existing message and file syntaxes, including: 492 * Cryptographic Message Syntax [RFC5652] defines a syntax used to 493 digitally sign, digest, authenticate, or encrypt arbitrary message 494 content. 496 * The.ZIP File Format specification [ZIPFILE] developed by Phil 497 Katz. 499 * The BitCoin Block chain [BLOCKCHAIN]. 501 * JSON Web Encryption and JSON Web Signature 503 Attempting to make use of these specifications in a layered fashion 504 would require at least three separate encoders and introduce 505 unnecessary complexity. Furthermore, there is considerable overlap 506 between the specifications providing multiple means of achieving the 507 same ends, all of which must be supported if decoders are to work 508 reliably. 510 1.3.7. Efficiency 512 Every data format represents a compromise between different concerns, 513 in particular: 515 Compactness The space required to record data in the encoding. 517 Memory Overhead The additional volatile storage (RAM) required to 518 maintain indexes etc. to support efficient retrieval operations. 520 Number of Operations The number of operations required to retrieve 521 data from or append data to an existing encoded sequence. 523 Number of Disk Seek Operations Optimizing the response time of 524 magnetic storage media to random access read requests has 525 traditionally been one of the central concerns of database design. 526 The DARE Sequence format is designed to the assumption that this 527 will cease to be a concern as solid state media replaces magnetic. 529 While the cost of storage of all types has declined rapidly over the 530 past decades, so has the amount of data to be stored. DARE Sequence 531 represents a pragmatic balance of these considerations for current 532 technology. In particular, since payload volumes are likely to be 533 very large, memory and operational efficiency are considered higher 534 priorities than compactness. 536 2. Definitions 538 2.1. Related Specifications 540 The DARE Envelope and Sequence formats are based on the following 541 existing standards and specifications. 543 Object serialization The JSON-B [draft-hallambaker-jsonbcd] encoding 544 is used for object serialization. This encoding is an extension 545 of the JavaScript Object Notation (JSON) [RFC7159]. 547 Message syntax The cryptographic processing model is based on JSON 548 Web Signature (JWS) [RFC7515], JSON Web Encryption (JWE) [RFC7516] 549 and JSON Web Key (JWK) [RFC7517]. 551 Cryptographic primitives. The HMAC-based Extract-and-Expand Key 552 Derivation Function [RFC5869] and Advanced Encryption Standard 553 (AES) Key Wrap with Padding Algorithm [RFC3394] are used. 555 The Uniform Data Fingerprint method of presenting data digests is 556 used for key identifiers and other purposes 557 [draft-hallambaker-mesh-udf]. 559 Cryptographic algorithms The cryptographic algorithms and 560 identifiers described in JSON Web Algorithms (JWA) [RFC7518] are 561 used together with additional algorithms as defined in the JSON 562 Object Signing and Encryption IANA registry [IANAJOSE]. 564 2.2. Requirements Language 566 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 567 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 568 document are to be interpreted as described in [RFC2119]. 570 2.3. Defined terms 572 The terms "Authentication Tag", "Content Encryption Key", "Key 573 Management Mode", "Key Encryption", "Direct Key Agreement", "Key 574 Agreement with Key Wrapping" and "Direct Encryption" are defined in 575 the JWE specification [RFC7516]. 577 The terms "Authentication", "Ciphertext", "Digital Signature", 578 "Encryption", "Initialization Vector (IV)", "Message Authentication 579 Code (MAC)", "Plaintext" and "Salt" are defined by the Internet 580 Security Glossary, Version 2 [RFC4949]. 582 Annotated Envelope A DARE Envelope that contains an Annotations 583 field with at least one entry. 585 Authentication Data A Message Authentication Code or authentication 586 tag. 588 Complete Envelope A DARE envelope that contains the key exchange 589 information necessary for the intended recipient(s) to decrypt it. 591 Detached Envelope A DARE envelope that does not contain the key 592 exchange information necessary for the intended recipient(s) to 593 decrypt it. 595 Encryption Context The master key, encryption algorithms and 596 associated parameters used to generate a set of one or more 597 enhanced data sequences. 599 Encoded data sequence (EDS) A sequence consisting of a salt, content 600 data and authentication data (if required by the encryption 601 context). 603 Enhancement Applying a cryptographic operation to a data sequence. 604 This includes encryption, authentication and both at the same 605 time. 607 Generator The party that generates a DARE envelope. 609 Group Encryption Key A key used to encrypt data to be read by a 610 group of users. This is typically achieved by means of some form 611 of proxy re-encryption or distributed key generation. 613 Group Encryption Key Identifier A key identifier for a group 614 encryption key. 616 Master Key (MK) The master secret from which keys are derived for 617 authenticating enhanced data sequences. 619 Recipient Any party that receives and processes at least some part 620 of a DARE envelope. 622 Related Envelope A set of DARE envelopes that share the same key 623 exchange information and hence the same Master Key. 625 Uniform Data Fingerprint (UDF) The means of presenting the result of 626 a cryptographic digest function over a data sequence and content 627 type identifier specified in the Uniform Data Fingerprint 628 specification [draft-hallambaker-mesh-udf] 630 3. DARE Envelope Architecture 632 A DARE Envelope is a sequence of three parts: 634 Header A JSON object containing information a reader requires to 635 begin processing the envelope. 637 Payload An array of octets. 639 Trailer A JSON object containing information calculated from the 640 envelope payload. 642 For example, the following sequence is a JSON encoded Envelope with 643 an empty header, a payload of zero length and an empty trailer: 645 [ {}, "", {} ] 647 DARE Envelopes MAY be encoded using JSON serialization or a binary 648 serialization for greater efficiency. 650 JSON [RFC7159] Offers compatibility with applications and libraries 651 that support JSON. Payload data is encoded using Base64 incurring 652 a 33% overhead. 654 JSON-B [draft-hallambaker-jsonbcd] A superset of JSON encoding that 655 permits binary data to be encoded as a sequence of length-data 656 segments. This avoids the Base64 overhead incurred by JSON 657 encoding. Since JSON-B is a superset of JSON encoding, an 658 application can use a single decoder for either format. 660 JSON-C [draft-hallambaker-jsonbcd] A superset of JSON-C which 661 provides additional efficiency by allowing field tags and other 662 repeated string data to be encoded by reference to a dictionary. 663 Since JSON-C is a superset of JSON and JSON-B encodings, an 664 application can use a single decoder for all three formats. 666 DARE Envelope processors MUST support JSON serialization and SHOULD 667 support JSON-B serialization. 669 3.1. Processing Considerations 671 The DARE Envelope Syntax supports single pass encoding and decoding 672 without buffering of data. All the information required to begin 673 processing a DARE envelope (key agreement information, digest 674 algorithms), is provided in the envelope header. All the information 675 that is derived from envelope processing (authentication codes, 676 digest values, signatures) is presented in the envelope trailer. 678 The choice of envelope encoding does not affect the semantics of 679 envelope processing. A DARE Envelope MAY be reserialized under the 680 same serialization or converted from any of the specified 681 serialization to any other serialization without changing the 682 semantics or integrity properties of the envelope. 684 3.2. Encoded Data Sequence 686 An encoded data sequence (EDS) is a sequence of octets that encodes a 687 data sequence according to cryptographic enhancements specified in 688 the context in which it is presented. An EDS MAY be encrypted and 689 MAY be authenticated by means of a MAC. The keys and other 690 cryptographic parameters used to apply these enhancements are derived 691 from the cryptographic context and a Salt prefix specified in the EDS 692 itself. 694 An EDS sequence contains exactly three binary fields encoded in 695 JSON-B serialization as follows: 697 Salt Prefix A sequence of octets used to derive the encryption key, 698 Initialization Vector and MAC key as required. 700 Body The plaintext or encrypted content. 702 Authentication Tag The authentication code value in the case that 703 the cryptographic context specifies use of authenticated 704 encryption or a MAC, otherwise is a zero-length field. 706 Requiring all three fields to be present, even in cases where they 707 are unnecessary simplifies processing at the cost of up to six 708 additional data bytes. 710 The encoding of the 'From' header of the previous example as a 711 plaintext EDS is as follows: 713 88 01 714 00 715 88 17 716 46 72 6f 6d 3a 20 41 6c 69 63 65 40 65 78 61 6d 717 70 6c 65 2e 63 6f 6d 718 [EOF] 720 3.3. Content Metadata and Annotations 722 A header MAY contain header fields describing the payload content. 723 These include: 725 ContentType Specifies the IANA Media Type [RFC6838]. 727 Annotations A list of Encoded Data Sequences that provide 728 application specific annotations to the envelope. 730 For example, consider the following mail message: 732 From: Alice@example.com 733 To: bob@example.com 734 Subject: TOP-SECRET Product Launch Today! 736 The CEO told me the product launch is today. Tell no-one! 738 Existing encryption approaches require that header fields such as the 739 subject line be encrypted with the body of the message or not 740 encrypted at all. Neither approach is satisfactory. In this 741 example, the subject line gives away important information that the 742 sender probably assumed would be encrypted. But if the subject line 743 is encrypted together with the message body, a mail client must 744 retrieve at least part of the message body to provide a 'folder' 745 view. 747 The plaintext form of the equivalent DARE Message encoding is: 749 [{ 750 "annotations":["iAEAiBdGcm9tOiBBbGljZUBleGFtcGxlLmNvbQ", 751 "iAEBiBNUbzogYm9iQGV4YW1wbGUuY29t", 752 "iAECiClTdWJqZWN0OiBUT1AtU0VDUkVUIFByb2R1Y3QgTGF1bmNoIFRvZG 753 F5IQ" 754 ], 755 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS 756 1tYWlsIn0"}, 757 "VGhlIENFTyB0b2xkIG1lIHRoZSBwcm9kdWN0IGxhdW5jaCBpcyB0b2RheS4gVG 758 VsbCBuby1vbmUh" 759 ] 761 This contains the same information as before but the data we might 762 wish to encrypt to protect the confidentiality of the payload is 763 separated from data required for processing. 765 3.4. Encryption and Integrity 767 Encryption and integrity protections MAY be applied to any DARE 768 Envelope Payload and Annotations. 770 The following is an encrypted version of the message shown earlier. 771 The payload and annotations have both increased in size as a result 772 of the block cipher padding. The header now includes Recipients and 773 Salt fields to enable the content to be decoded. 775 [{ 776 "enc":"A256CBC", 777 "kid":"EBQO-VVRO-PZGV-RRCY-HH2Q-E2SE-G4W6", 778 "Salt":"6LmVUEKI8-q72JYJg80NVQ", 779 "annotations":["iAEAiCD7pIbyRm2fJJn_9KraDZihW-F8Fn1iIaVBlq3lZL 780 4X4A", 781 "iAEBiCA9QH-N2ClnxAikU15FFjeHFG9H5qX6zrDujPGUL2s1Ig", 782 "iAECiDAHsjD13dra5_1NakdNKi6Rj8P7E6fMB_Y1UpL-CWdY0Yse1ibgZ4 783 97tlP8nKlMIq0" 784 ], 785 "recipients":[{ 786 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 787 "epk":{ 788 "PublicKeyECDH":{ 789 "crv":"Ed25519", 790 "Public":"OYT5iH4doxVrj90NRowmffE20OOPLlRGqaCav6b-Xw4"}}, 791 "wmk":"N3KQ0jcCztbOMSOwcvy_UdGNsLL-PMtd9_ZMuWqT4GzEIXj33a 792 HlKQ"} 793 ], 794 "ContentMetaData":"ewogICJjdHkiOiAiYXBwbGljYXRpb24vZXhhbXBsZS 795 1tYWlsIn0"}, 796 "bWhY3cljSPEQfl6k6jzZQRZR53Fbrbz1c9OwNp52Ry4_kbv961FXPO-j28kRms 797 eSrH6_hzJZTMOuse1KQA812w" 798 ] 800 For efficiency of processing, the ContentMetaData is presented in 801 plaintext. This header could be encrypted as an EDS sequence and 802 presented as a cloaked header. 804 3.4.1. Key Exchange 806 The DARE key exchange is based on the JWE key exchange except that 807 encryption modes are intentionally limited and the output of the key 808 exchange is the DARE Master Key rather than the Content Encryption 809 Key. 811 A DARE Key Exchange MAY contain any number of Recipient entries, each 812 providing a means of decrypting the Master Key using a different 813 private key. 815 If the Key Exchange mechanism supports message recovery, Direct Key 816 Agreement is used, in all other cases, Key Wrapping is used. 818 This approach allows envelopes with one intended recipient to be 819 handled in the exact same fashion as envelopes with multiple 820 recipients. While this does require an additional key wrapping 821 operation, that could be avoided if an envelope has exactly one 822 intended recipient, this is offset by the reduction in code 823 complexity. 825 If the key exchange algorithm does not support message recovery (e.g. 826 Diffie Hellman and Elliptic Curve Diffie-Hellman), the HKDF Extract- 827 and-Expand Key Derivation Function is used to derive a master key 828 using the following info tag: 830 "dare-master" [64 61 72 65 2d 6d 61 73 74 65 72] Key derivation info 831 field used when deriving a master key from the output of a key 832 exchange. 834 The master key length is the maximum of the key size of the 835 encryption algorithm specified by the key exchange header, the key 836 size of the MAC algorithm specified by the key exchange header (if 837 used) and 256. 839 3.4.2. Key Identifiers 841 The JWE/JWS specifications define a kid field for use as a key 842 identifier but not how the identifier itself is constructed. All 843 DARE key identifiers are either UDF key fingerprints 844 [draft-hallambaker-mesh-udf] or Mesh/Recrypt Group Key Identifiers. 846 A UDF fingerprint is formed as the digest of an IANA content type and 847 the digested data. A UDF key fingerprint is formed with the content 848 type application/pkix-keyinfo and the digested data is the ASN.1 DER 849 encoded PKIX certificate keyInfo sequence for the corresponding 850 public key. 852 A Group Key Identifier has the form @. Where 853 is a UDF key fingerprint and is the DNS 854 address of a service that provides the encryption service to support 855 decryption by group members. 857 3.4.3. Salt Derivation 859 A Master Salt is a sequence of 16 or more octets that is specified in 860 the Salt field of the header. 862 The Master Salt is used to derive salt values for the envelope 863 payload and associated encoded data sequences as follows. 865 Payload Salt = Master Salt 866 EDS Salt = Concatenate (Payload Salt Prefix, Master Salt) 868 Encoders SHOULD NOT generate salt values that exceed 1024 octets. 870 The salt value is opaque to the DARE encoding but MAY be used to 871 encode application specific semantics including: 873 * Frame number to allow reassembly of a data sequence split over a 874 sequence of envelopes which may be delivered out of order. 876 * Transmit the Master Key in the manner of a Kerberos ticket. 878 * Identify the Master Key under which the Enhanced Data Sequence was 879 generated. 881 * Enable access to the plaintext to be eliminated by erasure of the 882 encryption key. 884 For data erasure to be effective, the salt MUST be constructed so 885 that the difficulty of recovering the key is sufficiently high that 886 it is infeasible. For most purposes, a salt with 128 bits of 887 appropriately random data is sufficient. 889 3.4.4. Key Derivation 891 Encryption and/or authentication keys are derived from the Master Key 892 using a Extract-and-Expand Key Derivation Function as follows: 894 0. The Master Key and salt value are used to extract the PRK 895 (pseudorandom key) 897 1. The PRK is used to derive the algorithm keys using the 898 application specific information input for that key type. 900 The application specific information inputs are: 902 "dare-encrypt" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 903 encryption or encryption with authentication key. 905 "dare-iv" [64 61 72 65 2d 65 6e 63 72 79 70 74] To generate an 906 initialization vector. 908 "dare-mac" [dare-mac] To generate a Message Authentication Code key. 910 3.5. Signature 912 While encryption and integrity enhancements can be applied to any 913 part of a DARE Envelope, signatures are only applied to payload 914 digest values calculated over one or more envelope payloads. 916 The payload digest value for an envelope is calculated over the 917 binary payload data. That is, after any encryption enhancement has 918 been applied but before the envelope encoding is applied. This 919 allows envelopes to be converted from one encoding to another without 920 affecting signature verification. 922 Single Payload The signed value is the payload digest of the 923 envelope payload. 925 Multiple Payload. The signed value is the root of a Merkle Tree in 926 which the payload digest of the envelope is one of the leaves. 928 Verification of a multiple payload signature naturally requires the 929 additional digest values required to construct the Merkle Tree. 930 These are provided in the Trailer in a format that permits multiple 931 signers to reference the same tree data. 933 3.6. Algorithms 935 3.6.1. Field: kwd 937 The key wrapping and derivation algorithms. 939 Since the means of public key exchange is determined by the key 940 identifier of the recipient key, it is only necessary to specify the 941 algorithms used for key wrapping and derivation. 943 The default (and so far only) algorithm is kwd-aes-sha2-256-256. 945 Advanced Encryption Standard (AES) Key Wrap with Padding Algorithm 946 [RFC3394] is used to wrap the Master Exchange Key. AES 256 is used. 948 HMAC-based Extract-and-Expand Key Derivation Function [RFC5869] is 949 used for key derivation. SHA-2-256 is used for the hash function. 951 4. DARE Sequence Architecture 953 4.1. Sequence Navigation 955 Three means of locating frames in a container are supported: 957 Sequential Access frames sequentially starting from the start or the 958 end of the container. 960 Binary search Access any container frame by frame number in 961 _O(log_2(n)) time by means of a binary tree constructed while the 962 container is written._ 964 Index Access and container frame by frame number or by key by means 965 of an index record. 967 All DARE Sequences support sequential access. Only tree and Merkle 968 tree containers support binary search access. An index frame MAY be 969 written appended to any container and provides _O(1)_ access to any 970 frame listed in the index. 972 Two modes of compilation are considered: 974 Monolithic Frames are added to the container in a single operation, 975 e.g. file archives, 977 Incremental Additional frames are written to the container at 978 various intervals after it was originally created, e.g. server 979 logs, message spools. 981 In the monolithic mode, navigation requirements are best met by 982 writing an index frame to the end of the container when it is 983 complete. It is not necessary to construct a binary search tree 984 unless a Merkle tree integrity check is required. 986 In the incremental mode, Binary search provides an efficient means of 987 locating frames by frame number but not by key. Writing a complete 988 index to the container every _m_ write operations provides _O(m)_ 989 search access but requires _O(n^2) storage._ 991 Use of partial indexes provides a better compromise between speed and 992 efficiency. A partial index is written out every _m_ frames where 993 _m_ is a power of two. A complete index is written every time a 994 binary tree apex record is written. This approach provides for 995 _O(log_2(n)) search with incremental compilation with approximately 996 double the overhead of the monolithic case._ 998 4.1.1. Tree 1000 As previously described, the JBCD frame structure allows incremental 1001 navigation to the immediately preceding frame. The TreePosition 1002 parameter specifies the start position of _any_ previous frame in the 1003 container, thus allowing rapid navigation to that point. 1005 The TreePosition parameter MAY be used to enable any frame in the 1006 container to be retrieved in log_2(n) time by means of a binary 1007 search. The TreePosition parameter specifies the immediately 1008 preceding apex of a binary tree formed from the container entries. 1010 For example, the TreePosition of frame 6 in a container gives the 1011 location of frame 5, the TreePosition of frame 5 gives the location 1012 of frame 3, the TreePosition of frame 3 gives the location of frame 1013 1, and the TreePosition of frame 1 gives the location of frame 0: 1015 (Artwork only available as svg: No external link available, see 1016 draft-hallambaker-mesh-dare-15.html for artwork.) 1018 Figure 5: Binary search tree. 1020 An algorithm for efficiently calculating the immediately preceding 1021 apex is provided in Appendix C. 1023 4.1.2. Position Index 1025 Contains a table of frame number, position pairs pointing to prior 1026 locations in the file. 1028 4.1.3. Metadata Index 1030 Contains a list of IndexMeta entries. Each entry contains a metadata 1031 description and a list of frame indexes (not positions) of frames 1032 that match the description. 1034 4.2. Integrity Mechanisms 1036 Frame sequences in a DARE container MAY be protected against a frame 1037 insertion attack by means of a digest chain, a binary Merkle tree or 1038 both. 1040 4.2.1. Digest Chain calculation 1042 A digest chain is simple to implement but can only be verified if the 1043 full chain of values is known. Appending a frame to the chain has 1044 _O(1)_ complexity but verification has _O(n)_ complexity: 1046 (Artwork only available as svg: No external link available, see 1047 draft-hallambaker-mesh-dare-15.html for artwork.) 1049 Figure 6: Hash chain integrity check 1051 The value of the chain digest for the first frame (frame 0) is 1052 _H(H(null)+H(Payload_0)), where null is a zero length octet sequence 1053 and payloadn is the sequence of payload data bytes for frame n_ 1055 The value of the chain digest for frame _n_ is _H(H(Payload_(n-1) + 1056 H(Payloadn)), where A+B stands for concatenation of the byte 1057 sequences A and B._ 1059 4.2.2. Binary Merkle tree calculation 1061 The tree index mechanism describe earlier may be used to implement a 1062 binary Merkle tree. The value TreeDigest specifies the apex value of 1063 the tree for that node. 1065 Appending a frame to the chain has _O(log_2 (n)) complexity provided 1066 that the container format supports at least the binary tree index. 1067 Verifying a chain has O(log2 (n)) complexity, provided that the set 1068 of necessary digest inputs is known._ 1070 To calculate the value of the tree digest for a node, we first 1071 calculate the values of all the sub trees that have their apex at 1072 that node and then calculate the digest of that value and the 1073 immediately preceding local apex. 1075 4.2.3. Signature 1077 Payload data MAY be signed using a JWS [RFC7515] as applied in the 1078 Envelope. 1080 Signatures are specified by the Signatures parameter in the content 1081 header. The data that the signature is calculated over is defined by 1082 the typ parameter of the Signature as follows. 1084 Payload The value of the PayloadDigest parameter 1086 Chain The value of the ChainDigest parameter 1088 Tree The value of the TreeDigestFinal parameter 1090 If the typ parameter is absent, the value Payload is implied. 1092 A frame MAY contain multiple signatures created with the same signing 1093 key and different typ values. 1095 The use of signatures over chain and tree digest values permit 1096 multiple frames to be validated using a single signature verification 1097 operation. 1099 5. DARE Schema 1101 A DARE Envelope consists of a Header, an Enhanced Data Sequence (EDS) 1102 and an optional trailer. This section describes the JSON data fields 1103 used to construct headers, trailers and complete envelopes. 1105 Wherever possible, fields from JWE, JWS and JWK have been used. In 1106 these cases, the fields have the exact same semantics. Note however 1107 that the classes in which these fields are presented have different 1108 structure and nesting. 1110 5.1. Envelope Classes 1112 A DARE envelope contains a single DAREMessageSequence in either the 1113 JSON or Compact serialization as directed by the protocol in which it 1114 is applied. 1116 5.1.1. Structure: DareEnvelopeSequence 1118 A DARE envelope containing Header, EDS and Trailer in JSON object 1119 encoding. Since a DAREMessage is almost invariably presented in JSON 1120 sequence or compact encoding, use of the DAREMessage subclass is 1121 preferred. 1123 Although a DARE envelope is functionally an object, it is serialized 1124 as an ordered sequence. This ensures that the envelope header field 1125 will always precede the body in a serialization, this allowing 1126 processing of the header information to be performed before the 1127 entire body has been received. 1129 Header: DareHeader (Optional) The envelope header. May specify the 1130 key exchange data, pre-signature or signature data, cloaked 1131 headers and/or encrypted data sequences. 1133 Body: Binary (Optional) The envelope body 1135 Trailer: DareTrailer (Optional) The envelope trailer. If present, 1136 this contains the signature. 1138 5.2. Header and Trailer Classes 1140 A DARE sequence MUST contain a (possibly empty) DAREHeader and MAY 1141 contain a DARETrailer. 1143 5.2.1. Structure: DareTrailer 1145 A DARE envelope Trailer 1146 Signatures: DareSignature [0..Many] A list of signatures. A 1147 envelope trailer MUST NOT contain a signatures field if the header 1148 contains a signatures field. 1150 SignedData: Binary (Optional) Contains a DAREHeader object 1152 PayloadDigest: Binary (Optional) If present, contains the digest of 1153 the Payload. 1155 ChainDigest: Binary (Optional) If present, contains the digest of 1156 the PayloadDigest values of this frame and the frame immediately 1157 preceding. 1159 TreeDigest: Binary (Optional) If present, contains the Binary Merkle 1160 Tree digest value. 1162 5.2.2. Structure: DareHeader 1164 Inherits: DareTrailer 1166 A DARE Envelope Header. Since any field that is present in a trailer 1167 MAY be placed in a header instead, the envelope header inherits from 1168 the trailer. 1170 EnvelopeId: String (Optional) Unique identifier 1172 EncryptionAlgorithm: String (Optional) The encryption algorithm as 1173 specified in JWE 1175 DigestAlgorithm: String (Optional) Digest Algorithm. If specified, 1176 tells decoder that the digest algorithm is used to construct a 1177 signature over the envelope payload. 1179 KeyIdentifier: String (Optional) Base seed identifier. 1181 Salt: Binary (Optional) Salt value used to derrive cryptographic 1182 parameters for the content data. 1184 Malt: Binary (Optional) Hash of the Salt value used to derrive 1185 cryptographic parameters for the content data. This field SHOULD 1186 NOT be present if the Salt field is present. It is used to allow 1187 the salt value to be erased (thus rendering the payload content 1188 irrecoverable) without affecting the ability to calculate the 1189 payload digest value. 1191 Cloaked: Binary (Optional) If present in a header or trailer, 1192 specifies an encrypted data block containing additional header 1193 fields whose values override those specified in the envelope and 1194 context headers. 1196 When specified in a header, a cloaked field MAY be used to conceal 1197 metadata (content type, compression) and/or to specify an 1198 additional layer of key exchange. That applies to both the 1199 envelope body and to headers specified within the cloaked header. 1201 Processing of cloaked data is described in... 1203 EDSS: Binary [0..Many] If present, the Annotations field contains a 1204 sequence of Encrypted Data Segments encrypted under the envelope 1205 base seed. The interpretation of these fields is application 1206 specific. 1208 Signers: DareSignature [0..Many] A list of 'presignature' 1210 Recipients: DareRecipient [0..Many] A list of recipient key exchange 1211 information blocks. 1213 Policy: DarePolicy (Optional) A DARE security policy governing 1214 future additions to the container. 1216 ContentMetaData: Binary (Optional) If present contains a JSON 1217 encoded ContentInfo structure which specifies plaintext content 1218 metadata and forms one of the inputs to the envelope digest value. 1220 SequenceInfo: SequenceInfo (Optional) Information that describes 1221 container information 1223 SequenceIndex: SequenceIndex (Optional) An index of records in the 1224 current container up to but not including this one. 1226 Received: DateTime (Optional) Date on which the envelope was 1227 received. 1229 5.2.3. Structure: ContentMeta 1231 UniqueId: String (Optional) Unique object identifier 1233 Labels: String [0..Many] List of labels that are applied to the 1234 payload of the frame. 1236 KeyValues: KeyValue [0..Many] List of key/value pairs describing the 1237 payload of the frame. 1239 MessageType: String (Optional) The mesh message type 1240 ContentType: String (Optional) The content type field as specified 1241 in JWE 1243 Paths: String [0..Many] List of filename paths for the payload of 1244 the frame. 1246 Filename: String (Optional) The original filename under which the 1247 data was stored. 1249 Event: String (Optional) Operation on the header 1251 Created: DateTime (Optional) Initial creation date. 1253 Modified: DateTime (Optional) Date of last modification. 1255 Expire: DateTime (Optional) Date at which the associated transaction 1256 will expire 1258 First: Integer (Optional) Frame number of the first object instance 1259 value. 1261 Previous: Integer (Optional) Frame number of the immediately prior 1262 object instance value 1264 FileEntry: FileEntry (Optional) Information describing the file 1265 entry on disk. 1267 5.3. Cryptographic Data 1269 DARE envelope uses the same fields as JWE and JWS but with different 1270 structure. In particular, DARE envelopes MAY have multiple 1271 recipients and multiple signers. 1273 5.3.1. Structure: DareSignature 1275 The signature value 1277 Dig: String (Optional) Digest algorithm hint. Specifying the digest 1278 algorithm to be applied to the envelope body allows the body to be 1279 processed in streaming mode. 1281 Alg: String (Optional) Key exchange algorithm 1283 KeyIdentifier: String (Optional) Key identifier of the signature 1284 key. 1286 Certificate: X509Certificate (Optional) PKIX certificate of signer. 1288 Path: X509Certificate (Optional) PKIX certificates that establish a 1289 trust path for the signer. 1291 Manifest: Binary (Optional) The data description that was signed. 1293 SignatureValue: Binary (Optional) The signature value as an Enhanced 1294 Data Sequence under the envelope base seed. 1296 WitnessValue: Binary (Optional) The signature witness value used on 1297 an encrypted envelope to demonstrate that the signature was 1298 authorized by a party with actual knowledge of the encryption key 1299 used to encrypt the envelope. 1301 5.3.2. Structure: X509Certificate 1303 X5u: String (Optional) URL identifying an X.509 public key 1304 certificate 1306 X5: Binary (Optional) An X.509 public key certificate 1308 5.3.3. Structure: DareRecipient 1310 Recipient information 1312 KeyIdentifier: String (Optional) Key identifier for the encryption 1313 key. 1315 The Key identifier MUST be either a UDF fingerprint of a key or a 1316 Group Key Identifier 1318 KeyWrapDerivation: String (Optional) The key wrapping and derivation 1319 algorithms. 1321 WrappedBaseSeed: Binary (Optional) The wrapped base seed. The base 1322 seed is encrypted under the result of the key exchange. 1324 RecipientKeyData: String (Optional) The per-recipient key exchange 1325 data. 1327 5.3.4. Structure: DarePolicy 1329 EncryptionAlgorithm: String (Optional) The encryption algorithm to 1330 be used to compute the payload. 1332 DigestAlgorithm: String (Optional) The digest algorithm to be used 1333 to compute the payload digest. 1335 Encryption: String (Optional) The encryption policy specifier, 1336 determines how often a key exchange is required. 'Single': All 1337 entries are encrypted under the key exchange specified in the 1338 entry specifying this policy. 'Isolated': All entries are 1339 encrypted under a separate key exchange. 'All': All entries are 1340 encrypted. 'None': No entries are encrypted. 1342 Default value is 'None' if EncryptKeys is null, and 'All' 1343 otherwise. 1345 Signature: String (Optional) The signature policy 'None': No entries 1346 are signed. 'Last': The last entry in the container is signed. 1347 'Isolated': All entries are independently signed. 'Any': Entries 1348 may be signed. 1350 Default value is 'None' if SignKeys is null, and 'Any' otherwise. 1352 Sealed: Boolean (Optional) If true the policy is immutable and 1353 cannot be changed by a subsequent policy override. 1355 5.3.5. Structure: FileEntry 1357 Path: String (Optional) The file path in canonical form. 1359 CreationTime: DateTime (Optional) The creation time of the file on 1360 disk in UTC 1362 LastAccessTime: DateTime (Optional) The last access time of the file 1363 on disk in UTC 1365 LastWriteTime: DateTime (Optional) The last write time of the file 1366 on disk in UTC 1368 Attributes: Integer (Optional) The file attribues as a bitmapped 1369 integer. 1371 6. DARE Container Schema 1373 TBS stuff 1375 6.1. Container Headers 1377 TBS stuff 1379 6.1.1. Structure: SequenceInfo 1381 Information that describes container information 1383 DataEncoding: String (Optional) Specifies the data encoding for the 1384 header section of for the following frames. This value is ONLY 1385 valid in Frame 0 which MUST have a header encoded in JSON. 1387 ContainerType: String (Optional) Specifies the container type for 1388 the following records. This value is ONLY valid in Frame 0 which 1389 MUST have a header encoded in JSON. 1391 Index: Integer (Optional) The record index within the file. This 1392 MUST be unique and satisfy any additional requirements determined 1393 by the ContainerType. 1395 IsMeta: Boolean (Optional) If true, the current frame is a meta 1396 frame and does not contain a payload. 1398 Note: Meta frames MAY be present in any container. Applications 1399 MUST accept containers that contain meta frames at any position in 1400 the file. Applications MUST NOT interpret a meta frame as a data 1401 frame with an enpty payload. 1403 Default: Boolean (Optional) If set true in a persistent container, 1404 specifies that this record contains the default object for the 1405 container. 1407 TreePosition: Integer (Optional) Position of the frame containing 1408 the apex of the preceding sub-tree. 1410 IndexPosition: Integer (Optional) Specifies the position in the file 1411 at which the last index entry is to be found 1413 ExchangePosition: Integer (Optional) Specifies the position in the 1414 file at which the key exchange data is to be found 1416 6.2. Index Structures 1418 TBS stuff 1420 6.2.1. Structure: SequenceIndex 1422 A container index 1424 Full: Boolean (Optional) If true, the index is complete and contains 1425 position entries for all the frames in the file. If absent or 1426 false, the index is incremental and only contains position entries 1427 for records added since the last frame containing a 1428 ContainerIndex. 1430 Positions: IndexPosition [0..Many] List of container position 1431 entries 1433 6.2.2. Structure: IndexPosition 1435 Specifies the position in a file at which a specified record index is 1436 found 1438 Index: Integer (Optional) The record index within the file. 1440 Position: Integer (Optional) The record position within the file 1441 relative to the index base. 1443 UniqueId: String (Optional) Unique object identifier 1445 6.2.3. Structure: KeyValue 1447 Specifies a key/value entry 1449 Key: String (Optional) The key 1451 Value: String (Optional) The value corresponding to the key 1453 7. Dare Sequence Applications 1455 DARE Sequences are used to implement two forms of persistence store 1456 to support Mesh operations: 1458 Catalogs A set of related items which MAY be added, modified or 1459 deleted at any time. 1461 Spools A list of related items whose status MAY be changed at any 1462 time but which are immutable once added. 1464 Since DARE Sequences are an append only log format, entries can only 1465 be modified or deleted by adding items to the log to change the 1466 status of previous entries. It is always possible to undo any 1467 operation on a catalog or spool unless the underlying container is 1468 purged or the individual entries modified. 1470 7.1. Catalog 1472 Catalogs contain a set of entries, each of which is distinguished by 1473 a unique identifier. 1475 Three operations are supported: 1477 Add Addition of the entry to the catalog 1479 Update Modification of the data associated with the entry excluding 1480 the identifier 1482 Delete Removal of the entry from the catalog 1484 The set of valid state transitions is defined by the Finite State 1485 machine: 1487 (Add-Update*-Delete)* 1489 Catalogs are used to represent sets of persistent objects associated 1490 with a Mesh Service Account. The user's set of contacts for example. 1491 Each contact entry may be modified many times over time but refers to 1492 the same subject for its entire lifetime. 1494 7.2. Spool 1496 Spools contain lists of entries, each of which is distinguished by a 1497 unique identifier. 1499 Four operations are supported: 1501 Post Addition of the entry to the spool 1503 Processed Marks the entry as having been processed. 1505 Unprocessed Returns the entry to the unread state. 1507 Delete Mark the entry as deleted allowing recovery of associated 1508 storage in a subsequent purge operation. 1510 The set of valid state transitions is defined by the Finite State 1511 machine: 1513 Post-(Processed| Unprocessed| Delete *) 1515 Spools are used to represent time sequence ordered entries such as 1516 lists of messages being sent or received, task queues and transaction 1517 logs. 1519 7.3. Archive 1521 A DARE Archive is a DARE Sequence whose entries contain files. This 1522 affords the same functionality as a traditional ZIP or tar archive 1523 but with the added cryptographic capabilities provided by the DARE 1524 format. 1526 8. Future Work 1528 The current specification describes an approach in which containers 1529 are written according to a strict append-only policy. Greater 1530 flexibility may be achieved by loosening this requirement allowing 1531 record(s) at the end of the container to be overwritten. 1533 8.1. Terminal integrity check 1535 A major concern when operating a critical service is the possibility 1536 of a hardware or power failure occurring during a write operation 1537 causing the file update to be incomplete. While most modern 1538 operating systems have effective mechanisms in place to prevent 1539 corruption of the file system itself in such circumstances, this does 1540 not provide sufficient protection at the application level. 1542 Appending a null record containing a container-specific magic number 1543 provides an effective means of detecting this circumstance that can 1544 be quickly verified. 1546 If a container specifies a terminal integrity check value in the 1547 header of frame zero, the container is considered to be in an 1548 incomplete write state if the final frame is not a null record 1549 specifying the magic number. 1551 When appending new records to such containers, the old terminal 1552 integrity check record is overwritten by the data being added and a 1553 new integrity check record appended to the end. 1555 8.2. Terminal index record 1557 A writer can maintain a complete (or partial) index of the container 1558 in its final record without additional space overhead by overwriting 1559 the prior index on each update. 1561 8.3. Deferred indexing 1563 The task of updating terminal indexes may be deferred to a time when 1564 the machine is not busy. This improves responsiveness and may avoid 1565 the need to re-index containers receiving a sequence of updates. 1567 This approach may be supported by appending new entries to the end of 1568 the container in the usual fashion and maintaining a record of 1569 containers to be updated as a separate task. 1571 When updating the index on a container that has been updated in this 1572 fashion, the writer must ensure that no data is lost even if the 1573 process is interrupted. The use of guard records and other 1574 precautions against loss of state is advised. 1576 9. Security Considerations 1578 This section describes security considerations arising from the use 1579 of DARE in general applications. 1581 Additional security considerations for use of DARE in Mesh services 1582 and applications are described in the Mesh Security Considerations 1583 guide [draft-hallambaker-mesh-security]. 1585 9.1. Encryption/Signature nesting 1587 9.2. Side channel 1589 9.3. Salt reuse 1591 10. IANA Considerations 1593 11. Acknowledgements 1595 A list of people who have contributed to the design of the Mesh is 1596 presented in [draft-hallambaker-mesh-architecture]. 1598 The name Data At Rest Encryption was proposed by Melhi Abdulhayo?lu. 1600 12. Appendix A: DARE Envelope Examples and Test Vectors 1602 13. Test Examples 1604 In the following examples, Alice's encryption private key parameters 1605 are: 1607 { 1608 "PrivateKeyECDH":{ 1609 "crv":"Ed25519", 1610 "Private":"YX39DIWEBqxIJOBjR1yPNubhuB4oKmyz_f_PZk2Wft4"}} 1612 Alice's signature private key parameters are: 1614 { 1615 "PrivateKeyECDH":{ 1616 "crv":"Ed25519", 1617 "Private":"i-OxFUS-3GO0ftVIy3TYOAPWBjOnz3ZUNqLEdIx7stA"}} 1619 The body of the test message is the UTF8 representation of the 1620 following string: 1622 "This is a test long enough to require multiple blocks" 1624 The EDS sequences, are the UTF8 representation of the following 1625 strings: 1627 "Subject: Message metadata should be encrypted" 1628 "2018-02-01" 1630 13.1. Plaintext Message 1632 A plaintext message without associated EDS sequences is an empty 1633 header followed by the message body: 1635 { 1636 "DareEnvelope":[{}, 1637 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBs 1638 ZSBibG9ja3M" 1639 ]} 1641 13.2. Plaintext Message with EDS 1643 If a plaintext message contains EDS sequences, these are also in 1644 plaintext: 1646 { 1647 "DareEnvelope":[{ 1648 "annotations":["iAEAiC1TdWJqZWN0OiBNZXNzYWdlIG1ldGFkYXRhIHNo 1649 b3VsZCBiZSBlbmNyeXB0ZWQ", 1650 "iAEBiAoyMDE4LTAyLTAx" 1651 ]}, 1652 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBs 1653 ZSBibG9ja3M" 1654 ]} 1656 13.3. Encrypted Message 1658 The creator generates a base seed: 1660 E6 71 C9 D3 99 29 57 00 01 C5 DF 54 5C 81 1F C6 1661 75 BF 99 11 EC F9 92 BA 4B FF 5E B2 0A F4 E8 8A 1663 For each recipient of the message: 1665 The creator generates an ephemeral key: 1667 { 1668 "PrivateKeyECDH":{ 1669 "crv":"Ed25519", 1670 "Private":"v1EY8ZhRS0pkHVz94JNyFA_RPhfcsdxFjYoHWe2LN84"}} 1672 The key agreement value is calculated: 1674 29 53 00 38 88 B1 25 C0 4C D6 DC 4F D0 C3 BD EB 1675 B0 B9 97 E7 CD 95 A0 8E AF A6 FE FD 18 A0 65 58 1677 The key agreement value is used as the input to a HKDF key derivation 1678 function with the info parameter master to create the key used to 1679 wrap the base seed: 1681 6A EA FC 8C 42 63 BD 42 A5 CC 41 6C 3A 5F 45 EA 1682 5C C8 E1 8F CA 41 3B 5B B1 14 33 1D 08 08 38 2D 1684 The wrapped base seed is: 1686 37 72 90 D2 37 02 CE D6 CE 31 23 B0 72 FC BF 51 1687 D1 8D B0 B2 FE 3C CB 5D F7 F6 4C B9 6A 93 E0 6C 1688 C4 21 78 F7 DD A1 E5 29 1690 This information is used to calculate the Recipient information shown 1691 in the example below. 1693 To encrypt a message, we first generate a unique salt value: 1695 B0 5D 23 F4 DC C8 92 EE CE 02 F7 33 96 5F 23 37 1697 The base seed and salt value are used to generate the payload 1698 encryption key: 1700 09 A5 BE 8B F7 48 35 A6 F0 E3 07 3C 9D 47 B6 1B 1701 1B AD 48 BB 99 E3 2F 92 CC FF C6 3F EC 3B 37 6C 1703 Since AES is a block cipher, we also require an initializarion 1704 vector: 1706 69 85 92 7A 4F F3 A1 C7 7A 04 20 90 9F 43 0E E8 1708 The output sequence is the encrypted bytes: 1710 92 58 64 D4 AF 6A 3D C1 59 2F ED 2B 02 EE FA 53 1711 9A 21 F1 57 F9 29 B6 5C 9D 71 1F 69 87 3A FD F8 1712 26 28 54 99 12 83 07 AE 73 72 45 73 8D 0F 66 9F 1713 1B F8 B6 A8 F7 F9 10 14 4B C7 23 95 96 9C E7 A2 1715 Since the message is not signed, there is no need for a trailer. The 1716 completed message is: 1718 { 1719 "DareEnvelope":[{ 1720 "enc":"A256CBC", 1721 "kid":"EBQK-WRSP-WK6E-AVC5-2ZQI-PZ5K-2K6I", 1722 "Salt":"sF0j9NzIku7OAvczll8jNw", 1723 "recipients":[{ 1724 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 1725 "epk":{ 1726 "PublicKeyECDH":{ 1727 "crv":"Ed25519", 1728 "Public":"LE55Fn_dDQFZvzlJvYToyy-GmKcHD4zELw8U6WWZK 1729 qk"}}, 1730 "wmk":"Q6XR1OPMdCNSaJ2Vd-BsdWNkAl5SMm3Hho6uoPmWPfVoWzeg 1731 WF9boQ"} 1732 ]}, 1733 "klhk1K9qPcFZL-0rAu76U5oh8Vf5KbZcnXEfaYc6_fgmKFSZEoMHrnNyRXON 1734 D2afG_i2qPf5EBRLxyOVlpznog" 1735 ]} 1737 13.4. Signed Message 1739 Signed messages specify the digest algorithm to be used in the header 1740 and the signature value in the trailer. Note that the digest 1741 algorithm is not optional since it serves as notice that a decoder 1742 should digest the payload value to enable signature verification. 1744 { 1745 "DareEnvelope":[{ 1746 "dig":"S512"}, 1747 "VGhpcyBpcyBhIHRlc3QgbG9uZyBlbm91Z2ggdG8gcmVxdWlyZSBtdWx0aXBs 1748 ZSBibG9ja3M", 1749 { 1750 "signatures":[{ 1751 "alg":"S512", 1752 "kid":"MAR2-RQDD-TLR2-UQP5-YCU5-26Y3-YWCZ", 1753 "signature":"araqhHqBT0RZa5qlXgtBS0udR3jdtSvlUiqNdsUlzd 1754 mjmbqvt83WKGmcS07JvLufqfF9delLCK5LT4yuAlO8Cg"} 1755 ], 1756 "PayloadDigest":"raim8SV5adPbWWn8FMM4mrRAQCO9A2jZ0NZAnFXWlG 1757 0xF6sWGJbnKSdtIJMmMU_hjarlIPEoY3vy9UdVlH5KAg"} 1758 ]} 1760 13.5. Signed and Encrypted Message 1762 A signed and encrypted message is encrypted and then signed. The 1763 signer proves knowledge of the payload plaintext by providing the 1764 plaintext witness value. 1766 { 1767 "DareEnvelope":[{ 1768 "enc":"A256CBC", 1769 "dig":"S512", 1770 "kid":"EBQN-KLCB-QE3Z-S4IC-Y46H-MN7G-FVSC", 1771 "Salt":"9_yqbGfRxVimraQfhaIFZQ", 1772 "recipients":[{ 1773 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 1774 "epk":{ 1775 "PublicKeyECDH":{ 1776 "crv":"Ed25519", 1777 "Public":"OT13Sphcbw3rDkKJdjtf73V2Ji0WlelQnULdE88Aj 1778 20"}}, 1779 "wmk":"o9WQhJKLmnMG39XgxOMAruKNauSiKudfNvZnlFUUsj5YqTC_ 1780 eMpHXg"} 1781 ]}, 1782 "gYktKBTQrq_lcFbGcJp7qT-BciaMw5WZ4-oRnLiLcu-87pDrOETXNl8yWG_p 1783 g1zyDeyGkvcYPgHIGzZ3O4zGSg", 1784 { 1785 "signatures":[{ 1786 "alg":"S512", 1787 "kid":"MAR2-RQDD-TLR2-UQP5-YCU5-26Y3-YWCZ", 1788 "signature":"iI0aJ271ojzRuTln6nr6Q_HGQO15mehYlMZSmXznmZ 1789 iFkwB9tY-J-5ASmNhntFeUBeFJkocfGw225HHG9pdsBw", 1790 "witness":"G9x6MI7_vEAK9bLN97ZgzzZeXGO1A8bGY53CmwSkVwE"} 1791 ], 1792 "PayloadDigest":"2Nw9Ydgm-aEoc5i4TwqT8lAuruwEtpfnj3KeLZsqoI 1793 j5SpsR613RjlIhDhsd2p2Tb2QS8eXD73csYNnlmA8G1A"} 1794 ]} 1796 14. Appendix B: DARE Sequence Examples and Test Vectors 1798 The data payloads in all the following examples are identical, only 1799 the authentication and/or encryption is different. 1801 * Frame 1..n consists of 300 bytes being the byte sequence 00, 01, 1802 02, etc. repeating after 256 bytes. 1804 For conciseness, the raw data format is omitted for examples after 1805 the first, except where the data payload has been transformed, (i.e. 1806 encrypted). 1808 14.1. Simple sequence 1810 The following example shows a simple sequence with first frame and a 1811 single data frame: 1813 f4 91 1814 f0 8b 1815 f0 00 1816 f0 00 1817 91 f4 1818 f5 01 73 1819 f0 42 1820 f1 01 2c 1821 73 01 f5 1823 Since there is no integrity check, there is no need for trailer 1824 entries. The header values are: 1826 Frame 0 1828 { 1829 "DareHeader":{ 1830 "policy":{}, 1831 "ContentMetaData":"e30", 1832 "SequenceInfo":{ 1833 "DataEncoding":"JSON", 1834 "ContainerType":"List", 1835 "Index":0}}} 1837 [Empty trailer] 1839 Frame 1 1841 { 1842 "DareHeader":{ 1843 "ContentMetaData":"e30", 1844 "SequenceInfo":{ 1845 "Index":1}}} 1847 [Empty trailer] 1849 14.2. Payload and chain digests 1851 The following example shows a chain sequence with a first frame and 1852 three data frames. The headers of these frames is the same as before 1853 but the frames now have trailers specifying the PayloadDigest and 1854 ChainDigest values: 1856 Frame 0 1858 { 1859 "DareHeader":{ 1860 "dig":"S512", 1861 "policy":{}, 1862 "ContentMetaData":"e30", 1863 "SequenceInfo":{ 1864 "DataEncoding":"JSON", 1865 "ContainerType":"Chain", 1866 "Index":0}}} 1868 [Empty trailer] 1870 Frame 1 1872 { 1873 "DareHeader":{ 1874 "dig":"S512", 1875 "ContentMetaData":"e30", 1876 "SequenceInfo":{ 1877 "Index":1}}} 1879 { 1880 "DareTrailer":{ 1881 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1882 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1883 "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYV 1884 RVz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 1886 Frame 2 1888 { 1889 "DareHeader":{ 1890 "dig":"S512", 1891 "ContentMetaData":"e30", 1892 "SequenceInfo":{ 1893 "Index":2}}} 1895 { 1896 "DareTrailer":{ 1897 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1898 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1899 "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYV 1900 RVz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 1902 Frame 3 1903 { 1904 "DareHeader":{ 1905 "dig":"S512", 1906 "ContentMetaData":"e30", 1907 "SequenceInfo":{ 1908 "Index":3}}} 1910 { 1911 "DareTrailer":{ 1912 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1913 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1914 "ChainDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYV 1915 RVz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 1917 14.3. Merkle Tree 1919 The following example shows a chain sequence with a first frame and 1920 six data frames. The trailers now contain the TreePosition and 1921 TreeDigest values: 1923 Frame 0 1925 { 1926 "DareHeader":{ 1927 "dig":"S512", 1928 "policy":{}, 1929 "ContentMetaData":"e30", 1930 "SequenceInfo":{ 1931 "DataEncoding":"JSON", 1932 "ContainerType":"Merkle", 1933 "Index":0}}} 1935 [Empty trailer] 1937 Frame 1 1938 { 1939 "DareHeader":{ 1940 "dig":"S512", 1941 "ContentMetaData":"e30", 1942 "SequenceInfo":{ 1943 "Index":1, 1944 "TreePosition":0}}} 1946 { 1947 "DareTrailer":{ 1948 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1949 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1950 "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1951 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 1953 Frame 2 1955 { 1956 "DareHeader":{ 1957 "dig":"S512", 1958 "ContentMetaData":"e30", 1959 "SequenceInfo":{ 1960 "Index":2, 1961 "TreePosition":392}}} 1963 { 1964 "DareTrailer":{ 1965 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1966 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1967 "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kok 1968 kFnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}} 1970 Frame 3 1972 { 1973 "DareHeader":{ 1974 "dig":"S512", 1975 "ContentMetaData":"e30", 1976 "SequenceInfo":{ 1977 "Index":3, 1978 "TreePosition":392}}} 1980 { 1981 "DareTrailer":{ 1982 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 1983 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 1984 "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 1985 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 1987 Frame 4 1989 { 1990 "DareHeader":{ 1991 "dig":"S512", 1992 "ContentMetaData":"e30", 1993 "SequenceInfo":{ 1994 "Index":4, 1995 "TreePosition":1676}}} 1997 { 1998 "DareTrailer":{ 1999 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 2000 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2001 "TreeDigest":"vJ6ngNATvZcXSMALi5IUqzl1GBxBnTNVcC87VL_BhMRCbAv 2002 KSj8gs0VFgxxLkZ2myrtaDIwhHoswiTiBMLNWug"}} 2004 Frame 5 2006 { 2007 "DareHeader":{ 2008 "dig":"S512", 2009 "ContentMetaData":"e30", 2010 "SequenceInfo":{ 2011 "Index":5, 2012 "TreePosition":1676}}} 2014 { 2015 "DareTrailer":{ 2016 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 2017 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2018 "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 2019 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 2021 Frame 6 2022 { 2023 "DareHeader":{ 2024 "dig":"S512", 2025 "ContentMetaData":"e30", 2026 "SequenceInfo":{ 2027 "Index":6, 2028 "TreePosition":2963}}} 2030 { 2031 "DareTrailer":{ 2032 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 2033 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2034 "TreeDigest":"WgHlz3EHczVPqgtpc39Arv7CFIsCbFVsk8wg0j2qLlEfur9 2035 SZ0mdr65Ka-HF0Qx8gg_DAoiJwUrwADDXyxVJOg"}} 2037 14.4. Signed sequence 2039 The following example shows a tree sequence with a signature in the 2040 final record. The signing key parameters are: 2042 { 2043 "PrivateKeyECDH":{ 2044 "crv":"Ed25519", 2045 "Private":"i-OxFUS-3GO0ftVIy3TYOAPWBjOnz3ZUNqLEdIx7stA"}} 2047 The sequence headers and trailers are: 2049 Frame 0 2051 { 2052 "DareHeader":{ 2053 "dig":"S512", 2054 "policy":{}, 2055 "ContentMetaData":"e30", 2056 "SequenceInfo":{ 2057 "DataEncoding":"JSON", 2058 "ContainerType":"Merkle", 2059 "Index":0}}} 2061 [Empty trailer] 2063 Frame 1 2064 { 2065 "DareHeader":{ 2066 "dig":"S512", 2067 "ContentMetaData":"e30", 2068 "SequenceInfo":{ 2069 "Index":1, 2070 "TreePosition":0}}} 2072 { 2073 "DareTrailer":{ 2074 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 2075 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2076 "TreeDigest":"T7S1FcrgY3AaWD4L-t5W1K-3XYkPTcOdGEGyjglTD6yMYVR 2077 Vz9tn_KQc6GdA-P4VSRigBygV65OEd2Vv3YDhww"}} 2079 Frame 2 2081 { 2082 "DareHeader":{ 2083 "dig":"S512", 2084 "ContentMetaData":"e30", 2085 "SequenceInfo":{ 2086 "Index":2, 2087 "TreePosition":392}}} 2089 { 2090 "DareTrailer":{ 2091 "PayloadDigest":"8dyi62d7MDJlsLm6_w4GEgKBjzXBRwppu6qbtmAl6UjZ 2092 DlZeaWQlBsYhOu88-ekpNXpZ2iY96zTRI229zaJ5sw", 2093 "TreeDigest":"7fHmkEIsPkN6sDYAOLvpIJn5Dg3PxDDAaq-ll2kh8722kok 2094 kFnZQcYtjuVC71aHNXI18q-lPnfRkmwryG-bhqQ"}} 2096 14.5. Encrypted sequence 2098 The following example shows a sequence in which all the frame 2099 payloads are encrypted under the same base seed established in a key 2100 agreement specified in the first frame. 2102 Frame 0 2103 { 2104 "DareHeader":{ 2105 "enc":"A256CBC", 2106 "kid":"EBQA-6HXW-N2VL-QJGQ-FC2P-IOYT-WPC6", 2107 "Salt":"6OYJju7Gj8yqWR7mTJRgoQ", 2108 "recipients":[{ 2109 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 2110 "epk":{ 2111 "PublicKeyECDH":{ 2112 "crv":"Ed25519", 2113 "Public":"iSCixJ6PL_Hk5EI9VI7V2bRrVgdcdf8LL2jLZDJvgVM"}}, 2114 "wmk":"vlqR9frLpklV4uhWdpibspUHGbwCxdJFgRD8nHQihRdi246AEc 2115 kx2Q"} 2116 ], 2117 "policy":{ 2118 "enc":"none", 2119 "dig":"none", 2120 "EncryptKeys":[{ 2121 "PublicKeyECDH":{ 2122 "crv":"Ed25519", 2123 "Public":"elXlMJ7VH6j7i-tJsSDwwnbWHRU7wnQOnBrJZVHXiWo"}} 2124 ], 2125 "Sealed":true}, 2126 "ContentMetaData":"e30", 2127 "SequenceInfo":{ 2128 "DataEncoding":"JSON", 2129 "ContainerType":"List", 2130 "Index":0}}} 2132 [Empty trailer] 2134 Frame 1 2135 { 2136 "DareHeader":{ 2137 "enc":"A256CBC", 2138 "kid":"EBQM-WPVJ-LD2X-MEKW-XKL5-JUYV-6ES4", 2139 "Salt":"zdNLitcxA2_6kvUvMnr9EQ", 2140 "recipients":[{ 2141 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 2142 "epk":{ 2143 "PublicKeyECDH":{ 2144 "crv":"Ed25519", 2145 "Public":"hS8D7duh7bP2DBugpDUS-yUs0U40fqBsrS5NKqgGgTk"}}, 2146 "wmk":"BNwEbb9EX_3v0siNWZ5jBM3cZW67R6RI_EhA1P-FE5oy5C7eZ- 2147 YwUQ"} 2148 ], 2149 "ContentMetaData":"e30", 2150 "SequenceInfo":{ 2151 "Index":1}}} 2153 [Empty trailer] 2155 Frame 2 2157 { 2158 "DareHeader":{ 2159 "enc":"A256CBC", 2160 "kid":"EBQD-6DCH-W3U5-R7FQ-GVZK-UYXM-GIA4", 2161 "Salt":"QUg6uXAODywqpNC1wYAjZg", 2162 "recipients":[{ 2163 "kid":"MBUX-V4NE-VRJS-6NT7-6QKR-DE2W-QQBG", 2164 "epk":{ 2165 "PublicKeyECDH":{ 2166 "crv":"Ed25519", 2167 "Public":"I7G-lWAN78mny3E8VuklpznSL7GV-BH70yBQHMdtM4g"}}, 2168 "wmk":"p2k1W_K9MdBNXj6rJ08yZrXdPhQHDo1Jn1y8MOGOsKLTNdJxb3 2169 f4uQ"} 2170 ], 2171 "ContentMetaData":"e30", 2172 "SequenceInfo":{ 2173 "Index":2}}} 2175 [Empty trailer] 2177 Here are the sequence bytes. Note that the content is now encrypted 2178 and has expanded by 25 bytes. These are the salt (16 bytes), the AES 2179 padding (4 bytes) and the JSON-B framing (5 bytes). 2181 f5 02 ef 2182 f1 02 d8 2183 f0 10 2184 f0 00 2185 ef 02 f5 2186 f5 02 f9 2187 f1 01 c3 2188 f1 01 30 2189 f9 02 f5 2190 f5 02 f9 2191 f1 01 c3 2192 f1 01 30 2193 f9 02 f5 2195 The following example shows a sequence in which all the frame 2196 payloads are encrypted under separate key agreements specified in the 2197 payload frames. 2199 Frame 0 2201 { 2202 "DareHeader":{ 2203 "policy":{ 2204 "enc":"none", 2205 "dig":"none", 2206 "Sealed":true}, 2207 "ContentMetaData":"e30", 2208 "SequenceInfo":{ 2209 "DataEncoding":"JSON", 2210 "ContainerType":"List", 2211 "Index":0}}} 2213 [Empty trailer] 2215 Frame 1 2217 { 2218 "DareHeader":{ 2219 "ContentMetaData":"e30", 2220 "SequenceInfo":{ 2221 "Index":1}}} 2223 [Empty trailer] 2225 Frame 2 2226 { 2227 "DareHeader":{ 2228 "ContentMetaData":"e30", 2229 "SequenceInfo":{ 2230 "Index":2}}} 2232 [Empty trailer] 2234 15. Appendix C: Previous Frame Function 2236 public long PreviousFrame (long Frame) { 2237 long x2 = Frame + 1; 2238 long d = 1; 2240 while (x2 > 0) { 2241 if ((x2 & 1) == 1) { 2242 return x2 == 1 ? (d / 2) - 1 : Frame - d; 2243 } 2244 d = d * 2; 2245 x2 = x2 / 2; 2246 } 2247 return 0; 2248 } 2250 16. Appendix D: Outstanding Issues 2252 The following issues need to be addressed. 2254 +================+==========================================+ 2255 | Issue | Description | 2256 +================+==========================================+ 2257 | Signature | No examples are given of signing a | 2258 | | container. Need individual, chain, | 2259 | | tree. Leave notarized for notary draft. | 2260 +----------------+------------------------------------------+ 2261 | Indexing | No examples are given of indexing a | 2262 | | container | 2263 +----------------+------------------------------------------+ 2264 | Archive | Should include a file archive example | 2265 +----------------+------------------------------------------+ 2266 | File Path | Mention the file path security issue in | 2267 | | the security considerations | 2268 +----------------+------------------------------------------+ 2269 | Security | Write Security considerations | 2270 | Considerations | | 2271 +----------------+------------------------------------------+ 2272 | AES-GCM | Switch to using AES GCM in the examples | 2273 +----------------+------------------------------------------+ 2274 | KMAC | Switch to using KMAC for KDF | 2275 +----------------+------------------------------------------+ 2276 | Witness | Complete handling of witness values. | 2277 +----------------+------------------------------------------+ 2278 | Schema | Complete the schema documentation | 2279 +----------------+------------------------------------------+ 2281 Table 1 2283 17. Normative References 2285 [draft-hallambaker-jsonbcd] 2286 Hallam-Baker, P., "Binary Encodings for JavaScript Object 2287 Notation: JSON-B, JSON-C, JSON-D", Work in Progress, 2288 Internet-Draft, draft-hallambaker-jsonbcd-21, 5 August 2289 2021, . 2292 [draft-hallambaker-mesh-architecture] 2293 Hallam-Baker, P., "Mathematical Mesh 3.0 Part I: 2294 Architecture Guide", Work in Progress, Internet-Draft, 2295 draft-hallambaker-mesh-architecture-19, 25 October 2021, 2296 . 2299 [draft-hallambaker-mesh-security] 2300 Hallam-Baker, P., "Mathematical Mesh 3.0 Part IX Security 2301 Considerations", Work in Progress, Internet-Draft, draft- 2302 hallambaker-mesh-security-08, 20 September 2021, 2303 . 2306 [draft-hallambaker-mesh-udf] 2307 Hallam-Baker, P., "Mathematical Mesh 3.0 Part II: Uniform 2308 Data Fingerprint.", Work in Progress, Internet-Draft, 2309 draft-hallambaker-mesh-udf-15, 25 October 2021, 2310 . 2313 [IANAJOSE] "[Reference Not Found!]". 2315 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 2316 Requirement Levels", BCP 14, RFC 2119, 2317 DOI 10.17487/RFC2119, March 1997, 2318 . 2320 [RFC2315] Kaliski, B., "PKCS #7: Cryptographic Message Syntax 2321 Version 1.5", RFC 2315, DOI 10.17487/RFC2315, March 1998, 2322 . 2324 [RFC3394] Schaad, J. and R. Housley, "Advanced Encryption Standard 2325 (AES) Key Wrap Algorithm", RFC 3394, DOI 10.17487/RFC3394, 2326 September 2002, . 2328 [RFC4880] Callas, J., Donnerhacke, L., Finney, H., Shaw, D., and R. 2329 Thayer, "OpenPGP Message Format", RFC 4880, 2330 DOI 10.17487/RFC4880, November 2007, 2331 . 2333 [RFC4949] Shirey, R., "Internet Security Glossary, Version 2", 2334 FYI 36, RFC 4949, DOI 10.17487/RFC4949, August 2007, 2335 . 2337 [RFC5869] Krawczyk, H. and P. Eronen, "HMAC-based Extract-and-Expand 2338 Key Derivation Function (HKDF)", RFC 5869, 2339 DOI 10.17487/RFC5869, May 2010, 2340 . 2342 [RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type 2343 Specifications and Registration Procedures", BCP 13, 2344 RFC 6838, DOI 10.17487/RFC6838, January 2013, 2345 . 2347 [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data 2348 Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March 2349 2014, . 2351 [RFC7515] Jones, M., Bradley, J., and N. Sakimura, "JSON Web 2352 Signature (JWS)", RFC 7515, DOI 10.17487/RFC7515, May 2353 2015, . 2355 [RFC7516] Jones, M. and J. Hildebrand, "JSON Web Encryption (JWE)", 2356 RFC 7516, DOI 10.17487/RFC7516, May 2015, 2357 . 2359 [RFC7517] Jones, M., "JSON Web Key (JWK)", RFC 7517, 2360 DOI 10.17487/RFC7517, May 2015, 2361 . 2363 [RFC7518] Jones, M., "JSON Web Algorithms (JWA)", RFC 7518, 2364 DOI 10.17487/RFC7518, May 2015, 2365 . 2367 18. Informative References 2369 [BLOCKCHAIN] 2370 Chain.com, "Blockchain Specification". 2372 [Davis2001] 2373 Davis, D., "Defective Sign & Encrypt in S/MIME, PKCS#7, 2374 MOSS, PEM, PGP, and XML", May 2001. 2376 [RFC5652] Housley, R., "Cryptographic Message Syntax (CMS)", STD 70, 2377 RFC 5652, DOI 10.17487/RFC5652, September 2009, 2378 . 2380 [ZIPFILE] PKWARE Inc, "APPNOTE.TXT - .ZIP File Format 2381 Specification", October 2014.