idnits 2.17.1 draft-ietf-jmap-blob-03.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** There are 29 instances of too long lines in the document, the longest one being 14 characters in excess of 72. -- The draft header indicates that this document updates RFC8620, but the abstract doesn't seem to directly say this. It does mention RFC8620 though, so this could be OK. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year == Using lowercase 'not' together with uppercase 'MUST', 'SHALL', 'SHOULD', or 'RECOMMENDED' is not an accepted usage according to RFC 2119. Please use uppercase 'NOT' together with RFC 2119 keywords (if that is what you mean). Found 'MUST not' in this paragraph: Servers MUST apply any access controls such that if the authenticated user would be unable to discover the blobId by making queries, then this fact can't be discovered via a Blob/lookup. For example, if an Email exists in a Mailbox which the authenticated user does not have access to see, then that emailId MUST not be returned in a lookup for a blob which is referenced by that email. -- The document date (16 October 2021) is 916 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'String' is mentioned on line 384, but not defined == Missing Reference: 'CatenateSourceObject' is mentioned on line 183, but not defined == Missing Reference: 'Id' is mentioned on line 395, but not defined == Missing Reference: 'BlobInfo' is mentioned on line 401, but not defined == Unused Reference: 'RFC8621' is defined on line 624, but no explicit reference was found in the text Summary: 1 error (**), 0 flaws (~~), 7 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 JMAP B. Gondwana, Ed. 3 Internet-Draft Fastmail 4 Updates: 8620 (if approved) 16 October 2021 5 Intended status: Standards Track 6 Expires: 19 April 2022 8 JMAP Blob management extension 9 draft-ietf-jmap-blob-03 11 Abstract 13 The JMAP base protocol (RFC8620) provides the ability to upload and 14 download arbitrary binary data via HTTP POST and GET on defined 15 endpoint. This binary data is called a "Blob". 17 This extension adds additional ways to create and access Blobs, by 18 making inline method calls within a standard JMAP request. 20 This extension also adds a reverse lookup mechanism to discover where 21 blobs are referenced within other data types. 23 Status of This Memo 25 This Internet-Draft is submitted in full conformance with the 26 provisions of BCP 78 and BCP 79. 28 Internet-Drafts are working documents of the Internet Engineering 29 Task Force (IETF). Note that other groups may also distribute 30 working documents as Internet-Drafts. The list of current Internet- 31 Drafts is at https://datatracker.ietf.org/drafts/current/. 33 Internet-Drafts are draft documents valid for a maximum of six months 34 and may be updated, replaced, or obsoleted by other documents at any 35 time. It is inappropriate to use Internet-Drafts as reference 36 material or to cite them other than as "work in progress." 38 This Internet-Draft will expire on 19 April 2022. 40 Copyright Notice 42 Copyright (c) 2021 IETF Trust and the persons identified as the 43 document authors. All rights reserved. 45 This document is subject to BCP 78 and the IETF Trust's Legal 46 Provisions Relating to IETF Documents (https://trustee.ietf.org/ 47 license-info) in effect on the date of publication of this document. 48 Please review these documents carefully, as they describe your rights 49 and restrictions with respect to this document. Code Components 50 extracted from this document must include Simplified BSD License text 51 as described in Section 4.e of the Trust Legal Provisions and are 52 provided without warranty as described in the Simplified BSD License. 54 Table of Contents 56 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 57 2. Conventions Used In This Document . . . . . . . . . . . . . . 3 58 3. Addition to the Capabilities Object . . . . . . . . . . . . . 3 59 3.1. urn:ietf:params:jmap:blob . . . . . . . . . . . . . . . . 3 60 4. Blob Methods . . . . . . . . . . . . . . . . . . . . . . . . 4 61 4.1. Blob/set . . . . . . . . . . . . . . . . . . . . . . . . 4 62 4.1.1. create . . . . . . . . . . . . . . . . . . . . . . . 4 63 4.1.2. update . . . . . . . . . . . . . . . . . . . . . . . 6 64 4.1.3. destroy . . . . . . . . . . . . . . . . . . . . . . . 6 65 4.2. Blob/get . . . . . . . . . . . . . . . . . . . . . . . . 7 66 4.3. Blob/lookup . . . . . . . . . . . . . . . . . . . . . . . 9 67 5. Security considerations . . . . . . . . . . . . . . . . . . . 10 68 6. IANA considerations . . . . . . . . . . . . . . . . . . . . . 11 69 6.1. JMAP Capability registration for "blob" . . . . . . . . . 11 70 6.2. JMAP Error Codes Registration for "unknownDataType" . . . 11 71 6.3. Creation of "JMAP Data Types" Registry . . . . . . . . . 12 72 7. Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 73 8. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 14 74 9. Normative References . . . . . . . . . . . . . . . . . . . . 14 75 10. Informative References . . . . . . . . . . . . . . . . . . . 14 76 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 14 78 1. Introduction 80 Sometimes JMAP ([RFC8620]) interactions require creating a Blob and 81 then referencing it. In the same way that IMAP Literals ([RFC7888]) 82 were extended to reduce roundtrips for simple data, embedding simple 83 small blobs into the JMAP method stream can reduce roundtrips. 85 Likewise, when fetching an object, it can be useful to also fetch the 86 raw content of that object without a separate roundtrip. 88 Where JMAP is being proxied through a system which applies additional 89 access restrictions, it can be useful to be able to see where a blob 90 is referenced in order to decide whether to allow it to be 91 downloaded. 93 2. Conventions Used In This Document 95 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 96 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 97 "OPTIONAL" in this document are to be interpreted as described in BCP 98 14 [RFC2119] [RFC8174] when, and only when, they appear in all 99 capitals, as shown here. 101 3. Addition to the Capabilities Object 103 The capabilities object is returned as part of the JMAP Session 104 object; see [RFC8620], Section 2. 106 This document defines an additional capability URI. 108 3.1. urn:ietf:params:jmap:blob 110 This represents support for additional API methods on the Blob 111 datatype. 113 The value of this property in the JMAP session "capabilities" 114 property is an empty object. 116 The value of this property in an account's "accountCapabilities" 117 property is an object that MAY contain the following information on 118 server capabilities and permissions for that account: 120 * "maxSizeBlobSet": UnsignedInt - if set, gives the maximum size of 121 a blob that the server will allow you to create (including with 122 catenate). This SHOULD be the same as the RFC8620 value 123 maxSizeUpload. 125 * "maxCatenateItems": UnsignedInt - if set, gives the maximum number 126 of of CatenateSourceObjects allowed per creation in a Blob/set. 127 Servers SHOULD allow at least 64 items. 129 * "supportedTypeNames": [String] - an array of data type names that 130 are supported for Blob/lookup. 132 Example: 134 { 135 "capabilities": { 136 ..., 137 "urn:ietf:params:jmap:blob": {} 138 }, 139 "accounts": { 140 "A13842": { 141 ... 142 "accountCapabilities": { 143 "urn:ietf:params:jmap:blob": { 144 "maxSizeBlobSet": 50000000, 145 "maxCatenateItems": 100, 146 "supportedTypeNames" : [ 147 "Mailbox", 148 "Thread", 149 "Email" 150 ] 151 } 152 } 153 } 154 } 155 } 157 4. Blob Methods 159 A blob is a sequence of zero or more octets. 161 The JMAP base spec [RFC8620] defines the Blob/copy method, which is 162 unchanged by this specfication, and is selected by the 163 urn:ietf:params:jmap:core capability. 165 The following JMAP Methods are selected by the 166 urn:ietf:params:jmap:blob capability. 168 4.1. Blob/set 170 This is a standard JMAP set method. 172 4.1.1. create 174 *Properties:* 176 Exactly one of: 178 * data:asText: String|null 180 * data:asBase64: String|null 181 * data:asHex: String|null 183 * catenate: [CatenateSourceObject] _list of one or more octet 184 sources in order_ 186 Also: 188 * type: String|null _hint for media type of the data_ 190 Result is: * id: Id _the blobId_ * type: String|null _media type as 191 given in the creation (if any); or detected from content; or null_ * 192 size: UnsignedInt _as per RFC8620 - the size of the blob in Octets_ 194 Any other properties identical to those that would be returned in the 195 JSON response of the RFC8620 upload endpoint. 197 CatenateSourceObject: 199 Exactly one of: 201 * data:asText: String|null 203 * data:asBase64: String|null 205 * data:asHex: String|null 207 or a blobId source: 209 * blobId: Id 211 * offset: UnsignedInt|null _may be zero_ 213 * length: UnsignedInt|null _must not be zero_ 215 If null then offset is assumed to be zero. If null then length is 216 the remaining octets in the blob. If the range can not be fully 217 satisfied (i.e. extends past the end of the data in the blob) then 218 the catenate itself is invalid and results in a notCreated response 219 for this creation id. 221 If the data properties or catenate properties have any invalid 222 references or invalid data contained in them, the server MUST NOT 223 guess as to the user's intent, and MUST reject the creation and 224 return a notCreated response for that creation id. 226 Likewise, invalid data in the base64 or hex fields is an error and 227 must result in a notCreated response for this creation id. 229 It is legal to create a blob by calling catenate with a single 230 CatenateSourceObject. Please note that a catenate source can not 231 contain additional sub-catenates, only data or blob sources. 233 A server SHOULD accept at least 64 catenate items. 235 4.1.2. update 237 It is not possible to update a Blob, so any update will result in a 238 notUpdated response. 240 4.1.3. destroy 242 If an uploaded Blob is not referenced by any persistent object, the 243 server SHOULD destroy the object. Some systems use a content-based 244 ID for blobs, so the server MAY respond destroyed and yet that blobId 245 still exist with the same content. 247 Example: 249 Method Call: 251 [ "Blob/set", { 252 "accountId" : "account1", 253 "create" : { 254 "1": { 255 "data:asBase64": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABAQMAAAAl21bKA 256 AAAA1BMVEX/AAAZ4gk3AAAAAXRSTlN/gFy0ywAAAApJRE 257 FUeJxjYgAAAAYAAzY3fKgAAAAASUVORK5CYII=", 258 "type" : "image/png" 259 }, 260 }, 261 }, "R1" ] 263 Response: 265 [ "Blob/set", { 266 "accountId" : "account1", 267 "created" : { 268 "1": { 269 "id" : "G4c6751edf9dd6903ff54b792e432fba781271beb", 270 "type" : "image/png", 271 "size" : 95 272 }, 273 }, 274 }, "R1" ] 276 4.2. Blob/get 278 A standard JMAP get with two additional parameters: 280 * offset: UnsignedInt|null _start this many octets into the blob 281 data_ 283 * length: UnsignedInt|null _return at most this many octets of the 284 blob data_ 286 *Request Properties:* 288 Any of 290 * data:asText 292 * data:asBase64 294 * data:asHex 296 * data _selects data:asText if the content is valid UTF-8, or 297 data:asBase64_ 299 * size 301 If not given, properties defaults to data and size. 303 *Result Properties:* 305 * data:asText: String|null _the raw octets of the selected range if 306 they are valid utf-8, otherwise null_ 308 * data:asBase64: String _the base64 encoding of the selected range_ 310 * data:asHex: String _the octets of the selected range encoded as 311 lowercase hexadecimal (0-9, a-f)_ 313 * isEncodingProblem: Boolean _(default: false)_ 315 * isTruncated: Boolean _(default: false)_ 317 * size: UnsignedInt _the number of octets in the entire blob, 318 regardless of offset/length selectors_ 320 The size value is always the number of octets in the entire blob, 321 regardless of offset and length. 323 The data fields contain a representation of the octets within the 324 selected range that are present in the blob. If the octets selected 325 are not valid utf-8 (including truncating in the middle of a multi- 326 octet sequence) and data:asText or data is selected, then the key 327 isEncodingProblem is set to true and the data:asText value is null. 328 In the case where data was requested, this also causes data:asBase64 329 to be returned. 331 If the selected range requests data outside the blob (i.e. the 332 offset+length is larger than the blob) then the result is either just 333 the octets from the offset to the end of the blob, or an empty string 334 if the offset is past the end of the blob. Either way, the 335 isTruncated property in the result is set to true to tell the client 336 that the requested range could not be fully satisfied. 338 Example (a blob containing the string "The quick brown fox jumped 339 over the lazy dog!") 341 Method Call: 343 [ "Blob/get", { 344 "accountId" : "account1", 345 "ids" : [ "G6ec94756e3e046be78fcb33953b85b944e70673e", "not-a-blob" ], 346 "properties" : [ "data:asText", "data:asBase64", 347 "data:asHex", "size" ], 348 "offset" : 4, 349 "length" : 9 350 }, "R1" ] 352 Response: 354 [ "Blob/get", { 355 "accountId" : "account1", 356 "list" : [ 357 { 358 "id" : "G6ec94756e3e046be78fcb33953b85b944e70673e", 359 "data:asText" : "quick bro", 360 "data:asBase64" : "cXVpY2sgYnJvCg==", 361 "data:asHex" : "717569636b2062726f", 362 "size" : 46 363 } 364 ], 365 "notFound" : [ "not-a-blob" ] 366 }, "R1" ] 367 4.3. Blob/lookup 369 Given a list of blobIds, this method does a reverse lookup in each of 370 the provided type names to find the list of Ids within that data type 371 which reference the provided blob. 373 The definition of reference is somewhat loosely defined, but roughly 374 means "you could discover this blobId by looking inside this object", 375 for example if a Mailbox contains an Email which references the 376 blobId, then it references that blobId. Likewise for a Thread. 378 *Parameters* 380 * accountId: "Id" 382 The id of the account used for the call. 384 * types: [String] 386 A list of type names from the "JMAP Data Types" registry. Only names 387 for which "Can Reference Blobs" is true may be specified, and the 388 capability which defines each type must also be used by the overall 389 JMAP request in which this method is called. 391 If a type name is not known by the server, or the associated 392 capability has not been requested, then the server returns an 393 "unknownDataType" error. 395 * ids: [Id] 397 A list of blobId values to be looked for. 399 *Response* 401 * list: [BlobInfo] 403 A list of BlobInfo objects. 405 *BlobInfo Object* 407 * id: Id 409 The Blob Identifier. 411 * types: String[Id List] 413 A map from type name to list of Ids of that data type (e.g. the name 414 "Email" maps to a list of emailIds) 415 If a blob is not visible to a user at all, then the server SHOULD 416 return that blobId in the notFound array, however it may also return 417 an empty list for each type name, as it may not be able to know if 418 other data types do reference that blob. 420 e.g. 422 [ "Blob/lookup", { 423 "types": ["Mailbox", "Thread", "Email"], 424 "ids": ["Gd2f81008cf07d2425418f7f02a3ca63a8bc82003", 425 "not-a-blob"] 426 }, "R1" ] 428 Response: 430 [ "Blob/lookup", { 431 "list": [ 432 { 433 "id": "Gd2f81008cf07d2425418f7f02a3ca63a8bc82003", 434 "types": { 435 "Mailbox": ["M54e97373", Mcbe6b662"], 436 "Thread": ["T1530616e"], 437 "Email": ["E16e70a73eb4", "E84b0930cf16"] 438 } 439 } 440 ], 441 "notFound": ["not-a-blob"] 442 }, "R1"] 444 5. Security considerations 446 JSON parsers are not all consistent in handling non-UTF-8 data. JMAP 447 requires that all JSON data be UTF-8 encoded, so servers MUST either 448 return data:asBase64 or isEncodingProblem: true and modify the data 449 to be UTF-8 safe. 451 Servers MUST apply any access controls such that if the authenticated 452 user would be unable to discover the blobId by making queries, then 453 this fact can't be discovered via a Blob/lookup. For example, if an 454 Email exists in a Mailbox which the authenticated user does not have 455 access to see, then that emailId MUST not be returned in a lookup for 456 a blob which is referenced by that email. 458 If a server might sometimes return all names empty rather than 459 putting a blobId in the notFound response to a Blob/get, then the 460 server SHOULD always return the same type of response, regardless of 461 whether a blob exists but the user can't access it, or doesn't exist 462 at all. This avoids leaking information about the existence of the 463 blob. 465 The server MUST NOT trust that the data given to a Blob/set is a well 466 formed instance of the specified media type, and if the server 467 attempts to parse the given blob, only hardened parsers designed to 468 deal with arbitrary untrusted data should be used. The server SHOULD 469 NOT reject data on the grounds that it is not a valid specimen of the 470 stated type. 472 6. IANA considerations 474 6.1. JMAP Capability registration for "blob" 476 IANA is requested to register the "blob" JMAP Capability as follows: 478 Capability Name: urn:ietf:params:jmap:blob 480 Specification document: this document 482 Intended use: common 484 Change Controller: IETF 486 Security and privacy considerations: this document, Section XXX 488 6.2. JMAP Error Codes Registration for "unknownDataType" 490 IANA is requested to register the "unknownDataType" JMAP Error Code 491 as follows: 493 JMAP Error Code: unknownDataType 495 Intended use: common 497 Change Controller: IETF 499 Reference: this document 501 Description: The server does not recognise this data type, or the 502 capability to enable it was not present. 504 6.3. Creation of "JMAP Data Types" Registry 506 IANA is requested to create a new registry "JMAP Data Types" with the 507 initial content: 509 +================+=========+======+=====================================+=========+ 510 |Type Name |Can |Can |Capability |Reference| 511 | |Reference|use | | | 512 | |Blobs |for | | | 513 | | |State | | | 514 | | |Change| | | 515 +================+=========+======+=====================================+=========+ 516 |Core |No |No |urn:ietf:params:jmap:core |[RFC8620]| 517 +----------------+---------+------+-------------------------------------+---------+ 518 |PushSubscription|No |No |urn:ietf:params:jmap:core |[RFC8620]| 519 +----------------+---------+------+-------------------------------------+---------+ 520 |Mailbox |Yes |Yes |urn:ietf:params:jmap:mail |[RFC8621]| 521 +----------------+---------+------+-------------------------------------+---------+ 522 |Thread |Yes |Yes |urn:ietf:params:jmap:mail |[RFC8621]| 523 +----------------+---------+------+-------------------------------------+---------+ 524 |Email |Yes |Yes |urn:ietf:params:jmap:mail |[RFC8621]| 525 +----------------+---------+------+-------------------------------------+---------+ 526 |EmailDelivery |No |Yes |urn:ietf:params:jmap:mail |[RFC8621]| 527 +----------------+---------+------+-------------------------------------+---------+ 528 |SearchSnippet |No |No |urn:ietf:params:jmap:mail |[RFC8621]| 529 +----------------+---------+------+-------------------------------------+---------+ 530 |Identity |No |Yes |urn:ietf:params:jmap:submission |[RFC8621]| 531 +----------------+---------+------+-------------------------------------+---------+ 532 |EmailSubmission |No |Yes |urn:ietf:params:jmap:submission |[RFC8621]| 533 +----------------+---------+------+-------------------------------------+---------+ 534 |VacationResponse|No |Yes |urn:ietf:params:jmap:vacationresponse|[RFC8621]| 535 +----------------+---------+------+-------------------------------------+---------+ 536 |MDN |No |No |urn:ietf:params:jmap:mdn |[RFC9007]| 537 +----------------+---------+------+-------------------------------------+---------+ 539 Table 1 541 7. Changes 543 EDITOR: please remove this section before publication. 545 The source of this document exists on github at: 546 https://github.com/brong/draft-gondwana-jmap-blob/ 547 (https://github.com/brong/draft-gondwana-jmap-blob/) 549 *draft-ieft-jmap-blob-03*: 551 * added capabilities object 552 * renamed types to typeNames and matchedIds 554 * added details of how to handle non-UTF8 data and truncation in 555 Blob/get 557 * added isTruncated and isEncodingProblem to Blob/get to tell the 558 client if the request wasn't entirely satisfied. 560 *draft-ieft-jmap-blob-02*: 562 * fixed incorrect RFC number in reference and HTTP PUT -> POST, 563 thanks Ken. 565 * added acknowledgements section 567 * removed all 'datatype' text and changed to 'data type' or 'type 568 name' as appropriate (issue #1 proposal) 570 * expanded security considerations section and moved optional Blob/ 571 lookup empty case into Blob/lookup section 573 *draft-ieft-jmap-blob-01*: 575 * renamed 'datatypes' to 'types' to align with PushSubscription from 576 RFC8620. 578 * added example for Blob/get 580 * specified offset and length precisely 582 *draft-ieft-jmap-blob-00*: 584 * initial adoption as an IETF document, otherwise identical to 585 draft-gondwana-jmap-blob-02 587 *draft-gondwana-jmap-blob-02* 589 * renamed 'objects' to 'datatypes' 591 * specified Blob/lookup 593 * added IANA registry for datatypes 595 *draft-gondwana-jmap-blob-01* 597 * added an example 599 *draft-gondwana-jmap-blob-00* 600 * initial proposal 602 8. Acknowledgements 604 Alexey Melnikov, Ken Murchison, and the JMAP working group at the 605 IETF. 607 9. Normative References 609 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 610 Requirement Levels", BCP 14, RFC 2119, 611 DOI 10.17487/RFC2119, March 1997, 612 . 614 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 615 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 616 May 2017, . 618 10. Informative References 620 [RFC7888] Melnikov, A., Ed., "IMAP4 Non-synchronizing Literals", 621 RFC 7888, DOI 10.17487/RFC7888, May 2016, 622 . 624 [RFC8621] Jenkins, N. and C. Newman, "The JSON Meta Application 625 Protocol (JMAP) for Mail", RFC 8621, DOI 10.17487/RFC8621, 626 August 2019, . 628 [RFC8620] Jenkins, N. and C. Newman, "The JSON Meta Application 629 Protocol (JMAP)", RFC 8620, DOI 10.17487/RFC8620, July 630 2019, . 632 Author's Address 634 Bron Gondwana (editor) 635 Fastmail 636 Level 2, 114 William St 637 Melbourne VIC 3000 638 Australia 640 Email: brong@fastmailteam.com 641 URI: https://www.fastmail.com