idnits 2.17.1 draft-ietf-extra-imap-fetch-preview-10.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year -- The document date (September 30, 2020) is 1303 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 1 error (**), 0 flaws (~~), 1 warning (==), 1 comment (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 EXTRA M. Slusarz 3 Internet-Draft Open-Xchange Inc. 4 Intended status: Standards Track September 30, 2020 5 Expires: April 3, 2021 7 IMAP4 Extension: Message Preview Generation 8 draft-ietf-extra-imap-fetch-preview-10 10 Abstract 12 This document specifies an Internet Message Access Protocol (IMAP) 13 protocol extension allowing a client to request a server-generated 14 abbreviated text representation of message data useful as a 15 contextual preview of the entire message. 17 Status of This Memo 19 This Internet-Draft is submitted in full conformance with the 20 provisions of BCP 78 and BCP 79. 22 Internet-Drafts are working documents of the Internet Engineering 23 Task Force (IETF). Note that other groups may also distribute 24 working documents as Internet-Drafts. The list of current Internet- 25 Drafts is at https://datatracker.ietf.org/drafts/current/. 27 Internet-Drafts are draft documents valid for a maximum of six months 28 and may be updated, replaced, or obsoleted by other documents at any 29 time. It is inappropriate to use Internet-Drafts as reference 30 material or to cite them other than as "work in progress." 32 This Internet-Draft will expire on April 3, 2021. 34 Copyright Notice 36 Copyright (c) 2020 IETF Trust and the persons identified as the 37 document authors. All rights reserved. 39 This document is subject to BCP 78 and the IETF Trust's Legal 40 Provisions Relating to IETF Documents 41 (https://trustee.ietf.org/license-info) in effect on the date of 42 publication of this document. Please review these documents 43 carefully, as they describe your rights and restrictions with respect 44 to this document. Code Components extracted from this document must 45 include Simplified BSD License text as described in Section 4.e of 46 the Trust Legal Provisions and are provided without warranty as 47 described in the Simplified BSD License. 49 Table of Contents 51 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 52 2. Conventions Used In This Document . . . . . . . . . . . . . . 3 53 3. FETCH Data Item . . . . . . . . . . . . . . . . . . . . . . . 4 54 3.1. Command . . . . . . . . . . . . . . . . . . . . . . . . . 4 55 3.2. Response . . . . . . . . . . . . . . . . . . . . . . . . 4 56 3.3. Preview Text Format . . . . . . . . . . . . . . . . . . . 5 57 4. LAZY Priority Modifier . . . . . . . . . . . . . . . . . . . 5 58 4.1. LAZY . . . . . . . . . . . . . . . . . . . . . . . . . . 5 59 4.2. Client Implementation Advice . . . . . . . . . . . . . . 6 60 5. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 6 61 6. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 8 62 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8 63 8. Security Considerations . . . . . . . . . . . . . . . . . . . 9 64 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 9 65 9.1. Normative References . . . . . . . . . . . . . . . . . . 9 66 9.2. Informative References . . . . . . . . . . . . . . . . . 10 67 Appendix A. Change History (To be removed by RFC Editor before 68 publication) . . . . . . . . . . . . . . . . . . . . 10 69 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 13 70 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 13 72 1. Introduction 74 Many modern mail clients display small extracts of the body text as 75 an aid to allow a user to quickly decide whether they are interested 76 in viewing the full message contents. Mail clients implementing the 77 Internet Message Access Protocol [RFC3501] would benefit from a 78 standardized, consistent way to generate these brief textual previews 79 of messages. 81 Generation of a preview on the server has several benefits. First, 82 it allows consistent representation of previews across all clients. 83 This standardized display can reduce user confusion when using 84 multiple clients, as abbreviated message representations in clients 85 will show identical message contents. 87 Second, server-side preview generation is more efficient. A client- 88 based algorithm needs to issue, at a minimum, a FETCH BODYSTRUCTURE 89 command in order to determine which MIME [RFC2045] body part(s) 90 should be represented in the preview. Subsequently, at least one 91 FETCH BODY command may be needed to retrieve body data used in 92 preview generation. These FETCH commands cannot be pipelined since 93 the BODYSTRUCTURE query must be parsed on the client before the list 94 of parts to be retrieved via the BODY command(s) can be determined. 96 Additionally, it may be difficult to predict the amount of body data 97 that must be retrieved to adequately represent the part via a 98 preview, therefore requiring inefficient fetching of excessive data 99 in order to account for this uncertainty. For example, a preview 100 algorithm to display data contained in a text/html [RFC2854] part 101 will likely strip the markup tags to obtain textual content. 102 However, without fetching the entire content of the part, there is no 103 way to guarantee that sufficient non-tag content will exist unless 104 either 1) the entire part is retrieved or 2) an additional partial 105 FETCH is executed when the client determines that it does not possess 106 sufficient data from a previous partial FETCH to display an adequate 107 representation of the preview. 109 Finally, server generation allows caching in a centralized location. 110 Using server-generated previews allows global generation once per 111 message, and that preview can be cached for the retention period of 112 the source message. Retrieval of message data may be expensive 113 within a server, for example, so a server can be configured to reduce 114 its storage retrieval load by pre-generating preview data. 116 A server indicates support for this extension via the "PREVIEW" 117 capability name. 119 2. Conventions Used In This Document 121 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 122 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and 123 "OPTIONAL" in this document are to be interpreted as described in BCP 124 14 [RFC2119] [RFC8174] when, and only when, they appear in all 125 capitals, as shown here. 127 "User" is used to refer to a human user, whereas "client" refers to 128 the software being run by the user. 130 In examples, "C:" and "S:" indicate lines sent by the client and 131 server respectively. If a single "C:" or "S:" label applies to 132 multiple lines, then the line breaks between those lines are for 133 editorial clarity only and are not part of the actual protocol 134 exchange. 136 As with all IMAP extension documents, the case used in writing IMAP 137 protocol elements herein is chosen for editorial clarity, and 138 implementations must pay attention to the numbered rules at the 139 beginning of [RFC3501] Section 9. 141 3. FETCH Data Item 143 3.1. Command 145 To retrieve a preview for a message, the "PREVIEW" FETCH attribute is 146 used when issuing a FETCH command. 148 3.2. Response 150 The server returns a variable-length string that is the generated 151 preview for that message. This string is intended to be viewed by 152 the user as a contextual preview of the entire message, and is not 153 intended to be interpreted in any way by the client software. 155 Example: Retrieving preview information in a SELECTed mailbox 157 C: A1 FETCH 1 (PREVIEW) 158 S: * 1 FETCH (PREVIEW "Preview text!") 159 S: A1 OK FETCH complete. 161 A server SHOULD strive to generate the same string for a given 162 message for each request. However, since previews are understood to 163 be an approximation of the message data and not a canonical view of 164 its contents, a client MUST NOT assume that a message preview is 165 immutable for a given message. This relaxed requirement permits a 166 server to offer previews as an option without requiring potentially 167 burdensome storage and/or processing requirements to guarantee 168 immutability for a use case that does not require this strictness. 169 For example, the underlying IMAP server may change due to a system 170 software upgrade; an account's state information may be retained in 171 the migration but the new server may generate different PREVIEW text 172 than the old server. 174 It is possible that the server has determined that no meaningful 175 preview text can be generated for a particular message. Examples of 176 this involve encrypted messages, content types the server does not 177 support previews of, and other situations where the server is not 178 able to extract information for a preview. In such cases, the server 179 MUST return a zero-length string. Clients SHOULD NOT send another 180 FETCH for a preview for such messages. (As discussed previously, 181 preview data is not immutable so there is chance that at some point 182 in the future the server would be able to generate meaningful text. 183 However, this scenario is expected to be rare so a client should not 184 continually send out requests to try to capture this infrequent 185 occurrence.) 187 If the LAZY modifier is used, the server MAY return NIL for the 188 preview response, indicating that preview generation could not be 189 completed without causing undue delay. A server MUST NOT return NIL 190 to a FETCH PREVIEW request made without the LAZY modifier. 192 3.3. Preview Text Format 194 The generated preview text MUST be treated as text/plain [RFC2046] 195 media type data by the client. 197 The generated string MUST NOT be content transfer encoded and MUST be 198 encoded in UTF-8 [RFC3629]. The server SHOULD remove any formatting 199 markup and do whatever processing might be useful in rendering the 200 preview as plain text. 202 For purposes of this section, a "preview character" is defined as a 203 single UCS character encoded in UTF-8. Note: a single preview 204 character may compromise multiple octets, so any buffers implemented 205 to conform to the string limitations identified in this document 206 should be sized to prevent possible overflow errors. 208 The server SHOULD limit the length of the preview text to 200 preview 209 characters. This length should provide sufficient data to generally 210 support both various languages (and their different average word 211 lengths) and diverse client display size requirements. 213 The server MUST NOT output preview text longer than 256 preview 214 characters. 216 If the preview is not generated based on the body content of the 217 message, and the LANGUAGE [RFC5255] extension is supported by the 218 server, the preview text SHOULD be generated according to the 219 language rules that apply to human-readable text. For example, a 220 message that consists of a single image MIME part has no human- 221 readable text from which to generate preview information. Instead, 222 the server may wish to output a description that the message contains 223 an image and describe some attributes of the image, such as image 224 format, size, and filename. This descriptive text is not a product 225 of the message body itself but is rather auto-generated data by the 226 server, and should thus use the rules defined for human-readable text 227 described in the LANGUAGE extension (if supported on the server). 229 4. LAZY Priority Modifier 231 4.1. LAZY 233 The LAZY modifier directs the server to return the preview 234 representation only if that data can be returned without undue delay 235 to the client. 237 If this modifier is used, and the server is unable to return preview 238 data without undue delay, the server MUST return NIL as the preview 239 response. 241 The LAZY modifier MUST be implemented by any server that supports the 242 PREVIEW extension. 244 4.2. Client Implementation Advice 246 Upon opening a mailbox, a client generally performs a FETCH of 247 message details in order to create a listing to present to the user 248 (e.g. ENVELOPE data). Using this extension, a client may want to 249 additionally display preview information as part of this listing. 250 Quickly providing the base mailbox listing, with basic message 251 details, is the primary goal of this command as this is required to 252 allow the user to begin interacting with the mailbox. Preview data 253 is likely to be of secondary importance; it provides useful context, 254 but it is not necessary to perform message actions. A client can 255 load unavailable previews in the background and display them 256 asynchronously to the user as the preview data is provided by the 257 server. 259 In this scenario, the client would add the PREVIEW data item, with 260 the LAZY modifier, to the list of FETCH items needed to generate the 261 mailbox listing. This allows the server to advantageously return 262 preview data without blocking the primary goal of quickly returning 263 the basic message details used to generate the mailbox listing. 265 Once this initial FETCH is complete, the client can then issue FETCH 266 requests, without the LAZY modifier, to load the PREVIEW data item 267 for the messages in which preview data was not returned. It is 268 RECOMMENDED that these FETCH requests be issued in small batches, 269 e.g., 50 messages per FETCH command, since preview generation may be 270 expensive and a single large request may exceed server resource 271 limits. 273 See Example 2 for an implementation of this strategy. 275 A client SHOULD NOT continually issue LAZY PREVIEW FETCH commands in 276 a selected mailbox as the server is under no requirement to return 277 preview information for this command, which could lead to an 278 unnecessary waste of system and network resources. 280 5. Examples 281 Example 1: Requesting PREVIEW without LAZY modifier. 283 C: A1 CAPABILITY 284 S: * CAPABILITY IMAP4rev1 PREVIEW 285 S: A1 OK Capability command completed. 286 [...a mailbox is SELECTed...] 287 C: A2 FETCH 1 (RFC822.SIZE PREVIEW) 288 S: * 1 FETCH (RFC822.SIZE 5647 PREVIEW {200} 289 S: Lorem ipsum dolor sit amet, consectetur adipiscing elit. 290 S: Curabitur aliquam turpis et ante dictum, et pulvinar dui congue. 291 S: Maecenas hendrerit, lorem non imperdiet pellentesque, nulla 292 S: ligula nullam 293 S: ) 294 S: A2 OK FETCH complete. 296 Example 2: Requesting PREVIEW with LAZY modifier, to obtain previews 297 during initial mailbox listing if readily available; otherwise, load 298 previews in background. 300 C: D1 FETCH 1:4 (ENVELOPE PREVIEW (LAZY)) 301 S: * 1 FETCH (ENVELOPE ("Wed, 23 Sep 2020 15:03:11 +0000" [...]) 302 PREVIEW "Preview text for message 1.") 303 S: * 2 FETCH (PREVIEW "" ENVELOPE 304 ("Thu, 24 Sep 2020 12:17:23 +0000" [...])) 305 S: * 3 FETCH (ENVELOPE ("Fri, 25 Sep 2020 09:13:45 +0000" [...]) 306 PREVIEW NIL) 307 S: * 4 FETCH (ENVELOPE ("Sat, 26 Sep 2020 07:11:18 +0000" [...]) 308 PREVIEW NIL) 309 S: D1 OK FETCH completed. 310 [...Client has preview for message 1 and knows that message 2 has 311 a preview that is empty; only need to request preview of 312 messages 3 & 4 (e.g. in background)...] 313 C: D2 FETCH 3:4 (PREVIEW) 314 S: * 3 FETCH (PREVIEW {30} 315 S: Message data from message 3. 316 S: ) 317 S: * 4 FETCH (PREVIEW "Message 4 preview") 318 S: D2 OK Fetch completed. 320 Example 3: Retrieve preview information for search results within a 321 single mailbox. Use SEARCHRES [RFC5182] extension to save a round- 322 trip. 324 C: E1 CAPABILITY 325 S: * CAPABILITY IMAP4rev1 PREVIEW SEARCHRES 326 S: E1 OK Capability command completed. 327 [...a mailbox is SELECTed...] 328 C: E2 SEARCH RETURN (SAVE) FROM "FOO" 329 C: E3 FETCH $ (UID PREVIEW (LAZY)) 330 S: E2 OK SEARCH completed. 331 S: * 5 FETCH (UID 13 PREVIEW "Preview!") 332 S: * 9 FETCH (UID 23 PREVIEW NIL) 333 S: E3 OK FETCH completed. 334 [...Retrieve message 9 preview in background...] 335 C: E4 UID FETCH 23 (PREVIEW) 336 S: * 9 FETCH (UID 23 PREVIEW "Another preview!") 337 S: E4 OK FETCH completed. 339 6. Formal Syntax 341 The following syntax specification uses the augmented Backus-Naur 342 Form (BNF) as described in ABNF [RFC5234]. It includes definitions 343 from IMAP [RFC3501]. 345 capability =/ "PREVIEW" 347 fetch-att =/ "PREVIEW" [SP "(" preview-mod *(SP 348 preview-mod) ")"] 350 msg-att-dynamic =/ "PREVIEW" SP nstring 352 preview-mod = "LAZY" 354 7. IANA Considerations 356 IMAP4 [RFC3501] capabilities are registered by publishing a standards 357 track or IESG-approved experimental RFC. The registry is currently 358 located at: 360 http://www.iana.org/assignments/imap-capabilities 362 This document requests that IANA adds the "PREVIEW" capability to the 363 IMAP4 [RFC3501] capabilities registry. 365 8. Security Considerations 367 Implementation of this extension might enable denial-of-service 368 attacks against server resources, due to excessive memory or CPU 369 usage during preview generation or increased storage usage if preview 370 results are stored on the server after generation. In order to 371 mitigate such attacks, servers SHOULD log the client authentication 372 identity on FETCH PREVIEW operations in order to facilitate tracking 373 of abusive clients. 375 Servers MAY limit the resources that preview generation uses. Such 376 resource limitations might, in an extreme example, cause a server to 377 return a preview that is the empty string for a message that 378 otherwise would have had a non-empty preview. However, it is 379 recommended that at least some preview text be provided in this 380 situation, even if the quality of the preview is degraded. 382 Just as the messages they summarize, preview data may contain 383 sensitive information. If generated preview data is stored on the 384 server, e.g. for caching purposes, these previews MUST be protected 385 with equivalent authorization and confidentiality controls as the 386 source message. 388 9. References 390 9.1. Normative References 392 [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 393 Extensions (MIME) Part Two: Media Types", RFC 2046, 394 DOI 10.17487/RFC2046, November 1996, 395 . 397 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 398 Requirement Levels", BCP 14, RFC 2119, 399 DOI 10.17487/RFC2119, March 1997, 400 . 402 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 403 4rev1", RFC 3501, DOI 10.17487/RFC3501, March 2003, 404 . 406 [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 407 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 408 2003, . 410 [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax 411 Specifications: ABNF", STD 68, RFC 5234, 412 DOI 10.17487/RFC5234, January 2008, 413 . 415 [RFC5255] Newman, C., Gulbrandsen, A., and A. Melnikov, "Internet 416 Message Access Protocol Internationalization", RFC 5255, 417 DOI 10.17487/RFC5255, June 2008, 418 . 420 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 421 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, 422 May 2017, . 424 9.2. Informative References 426 [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail 427 Extensions (MIME) Part One: Format of Internet Message 428 Bodies", RFC 2045, DOI 10.17487/RFC2045, November 1996, 429 . 431 [RFC2854] Connolly, D. and L. Masinter, "The 'text/html' Media 432 Type", RFC 2854, DOI 10.17487/RFC2854, June 2000, 433 . 435 [RFC5182] Melnikov, A., "IMAP Extension for Referencing the Last 436 SEARCH Result", RFC 5182, DOI 10.17487/RFC5182, March 437 2008, . 439 Appendix A. Change History (To be removed by RFC Editor before 440 publication) 442 Changes from draft-slusarz-imap-fetch-snippet-00: 444 o Added standardized language to Section 2 regarding IMAP ABNF 445 conventions 447 o Changed draft name to draft-ietf-extra-imap-fetch-snippet-## 449 Changes from draft-ietf-extra-imap-fetch-snippet-00: 451 o Changed nomenclature from "snippet" to "preview" 453 o Changed draft name to draft-ietf-extra-imap-fetch-preview-## 455 o Update to RFC 8174 boilerplate 457 o Updated length requirements for PREVIEW=FUZZY 458 o Added preview-atom ABNF to limit use of "=" character 460 o UTF-8 is a normative reference 462 o Clarify that characters for purpose of length limitations are 463 defined as UCS characters as encoded by UTF-8 465 o Fix some incorrect literal lengths in examples 467 Changes from draft-ietf-extra-imap-fetch-preview-00: 469 o Updated postal address 471 o Added example to FETCH response section 473 o Added example on how LANGUAGE extension may influence preview 474 generation 476 o Added recommendation that only one LAZY FETCH be executed for a 477 message per mailbox 479 o Added request to create algorithm and modifier registries 481 o Added requirement that algorithm and modifier names conform to RFC 482 6648 484 o Added DoS attack info to security considerations 486 o Distinguish between NIL response and zero-length string 488 o Don't use deprecated "X-" convention in example 490 o Spelling and nits 492 Changes from draft-ietf-extra-imap-fetch-preview-01: 494 o Fix capability ABNF 496 o Removed CAPABILITY string for examples where it did not add 497 valuable context 499 o Altered preview data in examples to cover a variety of potential 500 server return scenarios 502 o Added "SHOULD be registered" language to algorithm names and 503 priority modifiers 505 Changes from draft-ietf-extra-imap-fetch-preview-02: 507 o Move Acknowledgments to un-numbered appendix 509 o Improved abstract text 511 o Consistently use "priority modifiers" instead of "modifiers" 513 o Update example to conform with RFC 3501 UID FETCH requirements 515 Changes from draft-ietf-extra-imap-fetch-preview-03: 517 o Remove preview modifier registry request 519 o Improve instructions for registration of algorithms 521 o Add storage information to security considerations 523 o Clarify parsing of algorithm list in FETCH command 525 o Clarify difference between NIL response and zero-length string 527 o Add normative reference for text/plain 529 o Add warning regarding buffers and multiple octet preview 530 characters 532 o Clarify how to handle preview data return when using an explicit 533 algorithm list 535 o Various editorial fixes 537 Changes from draft-ietf-extra-imap-fetch-preview-04: 539 o Make clear that preview caching is tied to retention period of the 540 source message 542 Changes from draft-ietf-extra-imap-fetch-preview-05: 544 o Clarify "zero-length string" preview data vs. NIL preview data 546 o MIME data -> media type 548 o Capability registration should not include the algorithm name 550 o Give example of how PREVIEW data might change over time 552 Changes from draft-ietf-extra-imap-fetch-preview-06: 554 o Change algorithm names to media types 555 o FUZZY algorithm changed to text/imap-fetch-preview 557 o Remove server broadcast of PREVIEW algorithm extensions from 558 capability 560 o Default, fallback algorithm in absence of client selection now 561 MUST be text/imap-fetch-preview 563 o LAZY modifier should work on default algorithm if no specific 564 algorithm is provided as an argument 566 Changes from draft-ietf-extra-imap-fetch-preview-07: 568 o Remove algorithm selection; PREVIEW always returns text in format 569 defined in Section 3.3 571 Changes from draft-ietf-extra-imap-fetch-preview-08: 573 o FETCH PREVIEW without LAZY modifier MUST NOT return NIL 575 o Improve client implementation advice for LAZY modifier 577 Changes from draft-ietf-extra-imap-fetch-preview-09: 579 o Clarified that string response is to be interpreted by user, not 580 the client 582 o Give example behavior of resource limitation 584 o Various editorial fixes 586 Acknowledgments 588 The author would like to thank the following people for their 589 comments and contributions to this document: Stephan Bosch, Bron 590 Gondwana, Teemu Huovila, Neil Jenkins, Steffen Lehmann, Barry Leiba, 591 Alexey Melnikov, Chris Newman, Pete Resnick, Jeff Sipek, Timo 592 Sirainen, Steffen Templin, and Aki Tuomi. 594 Author's Address 596 Michael M. Slusarz 597 Open-Xchange Inc. 598 530 Lytton Avenue 599 Palo Alto, California 94301 600 US 602 Email: michael.slusarz@open-xchange.com