idnits 2.17.1 draft-ietf-appsawg-multimailbox-search-04.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- == There are 1 instance of lines with non-RFC2606-compliant FQDNs in the document. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the IETF Trust and authors Copyright Line does not match the current year (Using the creation date from RFC4466, updated by this document, for RFC5378 checks: 2005-03-09) -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 07, 2014) is 3550 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'LIMIT' is mentioned on line 307, but not defined ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 1 error (**), 0 flaws (~~), 3 warnings (==), 2 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Applications Area Working Group B. Leiba 3 Internet-Draft Huawei Technologies 4 Updates: 4466 (if approved) A. Melnikov 5 Obsoletes: 6237 (if approved) Isode Limited 6 Intended status: Standards Track August 07, 2014 7 Expires: February 06, 2015 9 IMAP4 Multimailbox SEARCH Extension 10 draft-ietf-appsawg-multimailbox-search-04 12 Abstract 14 The IMAP4 specification allows the searching of only the selected 15 mailbox. A user often wants to search multiple mailboxes, and a 16 client that wishes to support this must issue a series of SELECT and 17 SEARCH commands, waiting for each to complete before moving on to the 18 next. This extension allows a client to search multiple mailboxes 19 with one command, limiting the delays caused by many round trips, and 20 not requiring disruption of the currently selected mailbox. This 21 extension also uses MAILBOX, UIDVALIDITY, and TAG fields in ESEARCH 22 responses, allowing a client to pipeline the searches if it chooses. 23 This document updates RFC 4466 and obsoletes RFC 6237. 25 Status of this Memo 27 This Internet-Draft is submitted in full conformance with the 28 provisions of BCP 78 and BCP 79. 30 Internet-Drafts are working documents of the Internet Engineering 31 Task Force (IETF). Note that other groups may also distribute 32 working documents as Internet-Drafts. The list of current Internet- 33 Drafts is at http://datatracker.ietf.org/drafts/current/. 35 Internet-Drafts are draft documents valid for a maximum of six months 36 and may be updated, replaced, or obsoleted by other documents at any 37 time. It is inappropriate to use Internet-Drafts as reference 38 material or to cite them other than as "work in progress." 40 This Internet-Draft will expire on February 06, 2015. 42 Copyright Notice 44 Copyright (c) 2014 IETF Trust and the persons identified as the 45 document authors. All rights reserved. 47 This document is subject to BCP 78 and the IETF Trust's Legal 48 Provisions Relating to IETF Documents (http://trustee.ietf.org/ 49 license-info) in effect on the date of publication of this document. 50 Please review these documents carefully, as they describe your rights 51 and restrictions with respect to this document. Code Components 52 extracted from this document must include Simplified BSD License text 53 as described in Section 4.e of the Trust Legal Provisions and are 54 provided without warranty as described in the Simplified BSD License. 56 Table of Contents 58 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2 59 1.1. Conventions Used in This Document . . . . . . . . . . . . . 3 60 2. New ESEARCH Command . . . . . . . . . . . . . . . . . . . . . 3 61 2.1. The ESEARCH Response . . . . . . . . . . . . . . . . . . . . 4 62 2.2. Source Options: Specifying Mailboxes to Search . . . . . . . 5 63 2.3. Strictness in Search Matches . . . . . . . . . . . . . . . . 6 64 2.4. Server-Side Limitations on Search Volume . . . . . . . . . . 6 65 3. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 66 4. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 7 67 5. Security Considerations . . . . . . . . . . . . . . . . . . . 8 68 6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 69 7. Implementation Status . . . . . . . . . . . . . . . . . . . . 9 70 8. Changes Since RFC 6237 . . . . . . . . . . . . . . . . . . . . 10 71 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 72 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 10 73 10.1. Normative References . . . . . . . . . . . . . . . . . . . 10 74 10.2. Informative References . . . . . . . . . . . . . . . . . . 11 75 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 11 77 1. Introduction 79 The IMAP4 specification allows the searching of only the selected 80 mailbox. A user often wants to search multiple mailboxes, and a 81 client that wishes to support this must issue a series of SELECT and 82 SEARCH commands, waiting for each to complete before moving on to the 83 next. The commands can't be pipelined, because the server might run 84 them in parallel, and the untagged SEARCH responses could not then be 85 distinguished from each other. 87 This extension allows a client to search multiple mailboxes with one 88 command, and includes MAILBOX and TAG fields in the ESEARCH response, 89 yielding the following advantages: 91 o A single command limits the number of round trips needed to search 92 a set of mailboxes. 94 o A single command eliminates the need to wait for one search to 95 complete before starting the next. 97 o A single command allows the server to optimize the search, if it 98 can. 100 o A command that is not dependent upon the selected mailbox 101 eliminates the need to disrupt the selection state or to open 102 another IMAP connection. 104 o The MAILBOX, UIDVALIDITY, and TAG fields in the responses allow a 105 client to distinguish which responses go with which search (and 106 which mailbox). A client can safely pipeline these search 107 commands without danger of confusion. The addition of the MAILBOX 108 and UIDVALIDITY fields updates the search-correlator item defined 109 in [RFC4466]. 111 This extension was previously published as experimental. There is 112 now implementation experience, giving confidence in the protocol, so 113 this document puts the extension on the Standards Track, with some 114 minor updates that were informed by the implementation experience. A 115 brief summary of changes is in Section 8. 117 1.1. Conventions Used in This Document 119 In examples, "C:" indicates lines sent by a client that is connected 120 to a server. "S:" indicates lines sent by the server to the client. 122 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 123 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 124 document are to be interpreted as described in RFC 2119 [RFC2119]. 126 2. New ESEARCH Command 128 Arguments: OPTIONAL source options 129 OPTIONAL result options 130 OPTIONAL charset specification (see [RFC2978]) 131 searching criteria (one or more) 133 Responses: REQUIRED untagged response: ESEARCH 135 Result: OK -- search completed 136 NO -- error: cannot search that charset or criteria 137 BAD -- command unknown or arguments invalid 139 This section defines a new ESEARCH command, which works similarly to 140 the UID SEARCH command described in Section 2.6.1 of [RFC4466] 141 (initially described in Section 6.4.4 of [RFC3501] and extended by 142 [RFC4731]). 144 The ESEARCH command further extends searching by allowing for 145 optional source and result options. This document does not define 146 any new result options (see Section 3.1 of [RFC4731]). A server that 147 supports this extension includes "MULTISEARCH" in its IMAP capability 148 string. 150 Because there has been confusion about this, it is worth pointing out 151 that with ESEARCH, as with any SEARCH or UID SEARCH command, it MUST 152 NOT be considered an error if the search terms include a range of 153 message numbers that extends (or, in fact, starts) beyond the end of 154 the mailbox. For example, a client might want to establish a rolling 155 window through the search results this way: 157 C: tag1 UID ESEARCH FROM "frobozz" 1:100 159 ...followed later by this: 161 C: tag1 UID ESEARCH FROM "frobozz" 101:200 163 ...and so on. This tells the server to match only the first hundred 164 messages in the mailbox the first time, the second hundred the second 165 time, etc. In fact, it might likely allow the server to optimize the 166 search significantly. In the above example, whether the mailbox 167 contains 50 or 150 or 250 messages, neither of the search commands 168 shown will result in an error. It is up to the client to know when 169 to stop moving its search window. 171 2.1. The ESEARCH Response 173 In response to an ESEARCH command, the server MUST return ESEARCH 174 responses [RFC4731] (that is, not SEARCH responses). Because message 175 numbers are not useful for mailboxes that are not selected, the 176 responses MUST contain information about UIDs, not message numbers. 177 This is true even if the source options specify that only the 178 selected mailbox be searched. 180 Presence of a source option in the absence of a result option implies 181 the "ALL" result option (see Section 3.1 of [RFC4731]). Note that 182 this is not the same as the result from the SEARCH command described 183 in the IMAP base protocol [RFC3501]. 185 Source options describe which mailboxes must be searched for 186 messages. An ESEARCH command with source options does not affect 187 which mailbox, if any, is currently selected, regardless of which 188 mailboxes are searched. 190 For each mailbox satisfying the source options, a single ESEARCH 191 response MUST be returned if any messages in that mailbox match the 192 search criteria. An ESEARCH response MUST NOT be returned for 193 mailboxes that contain no matching messages. This is true even when 194 result options such as MIN, MAX, and COUNT are specified (see Section 195 3.1 of [RFC4731]), and the values returned (lowest UID matched, 196 highest UID matched, and number of messages matched, respectively) 197 apply to the mailbox reported in that ESEARCH response. 199 Note that it is possible for an ESEARCH command to return no untagged 200 responses (no ESEARCH responses at all), in the case that there are 201 no matches to the search in any of the mailboxes that satisfy the 202 source options. Clients can detect this situation by finding the 203 tagged OK response without having received any matching untagged 204 ESEARCH responses. 206 Each ESEARCH response MUST contain the MAILBOX, TAG, and UIDVALIDITY 207 correlators. Correlators allow clients to issue several ESEARCH 208 commands at once (pipelined). If the SEARCHRES [RFC5182] extension 209 is used in an ESEARCH command, that ESEARCH command MUST be executed 210 by the server after all previous SEARCH/ESEARCH commands have 211 completed and before any subsequent SEARCH/ESEARCH commands are 212 executed. The server MAY perform consecutive ESEARCH commands in 213 parallel as long as none of them use the SEARCHRES extension. 215 2.2. Source Options: Specifying Mailboxes to Search 217 The source options, if present, MUST contain a mailbox specifier as 218 defined in the IMAP NOTIFY extension [RFC5465], Section 6 (using the 219 "filter-mailboxes" ABNF item), with the following differences: 221 1. The "selected-delayed" specifier is not valid here. 223 2. A "subtree-one" specifier is added. The "subtree" specifier 224 results in a search of the specified mailbox and all selectable 225 mailboxes that are subordinate to it, through an indefinitely 226 deep hierarchy. The "subtree-one" specifier results in a search 227 of the specified mailbox and all selectable child mailboxes, one 228 hierarchy level down. 230 If "subtree" is specified, the server MUST defend against loops in 231 the hierarchy (for example, those caused by recursive file-system 232 links within the message store). The server SHOULD do this by 233 keeping track of the mailboxes that have been searched, and 234 terminating the hierarchy traversal when a repeat is found. If it 235 cannot do that, it MAY do it by limiting the hierarchy depth. 237 If the source options are not present, the value "selected" is 238 assumed -- that is, only the currently selected mailbox is searched. 240 The "personal" source option is a particularly convenient way to 241 search all of the current user's mailboxes. Note that there is no 242 way to use wildcard characters to search all mailboxes; the 243 "mailboxes" source option does not do wildcard expansion. 245 If the source options include (or default to) "selected", the IMAP 246 session MUST be in "selected" state. If the source options specify 247 other mailboxes and NOT "selected", then the IMAP session MUST be in 248 either "selected" or "authenticated" state. If the session is not in 249 a correct state, the ESEARCH command MUST return a "BAD" result. 251 The client SHOULD NOT provide source options that resolve to 252 including the same mailbox more than once. A server can, of course, 253 remove the duplicates before processing, but the server MAY return 254 "BAD" to an ESEARCH command with duplicate source mailboxes. 256 If the server supports the SEARCHRES [RFC5182] extension, then the 257 "SAVE" result option is valid only if "selected" is specified or 258 defaulted as the sole mailbox to be searched. If any source option 259 other than "selected" is specified, the ESEARCH command MUST return a 260 "BAD" result. 262 If the server supports the CONTEXT=SEARCH and/or CONTEXT=SORT 263 extension [RFC5267], then the following additional rules apply: 265 o The CONTEXT return option (Section 4.2 of [RFC5267]) can be used 266 with an ESEARCH command. 268 o If the UPDATE return option is used (Section 4.3 of [RFC5267]), it 269 MUST apply only to the currently selected mailbox. If UPDATE is 270 used and there is no mailbox currently selected, the ESEARCH 271 command MUST return a "BAD" result. 273 o The PARTIAL search return option (Section 4.4 of [RFC5267]) can be 274 used and applies to each mailbox searched by the ESEARCH command. 276 If the server supports the Access Control List (ACL) [RFC4314] 277 extension, then the logged-in user is required to have the "r" right 278 for each mailbox she wants to search. In addition, any mailboxes 279 that are not explicitly named (accessed through "personal" or 280 "subtree", for example) are required to have the "l" right. 281 Mailboxes matching the source options for which the logged-in user 282 lacks sufficient rights MUST be ignored by the ESEARCH command 283 processing. In particular, ESEARCH responses MUST NOT be returned 284 for those mailboxes. 286 2.3. Strictness in Search Matches 288 The base IMAP SEARCH command (Section 6.4.4. of [RFC3501]) requires 289 strict substring matching in text searches. Many servers, however, 290 use search engines that match strings in different ways, matching, 291 for example, "swim" to "swam" and "swum" as well, or only doing full 292 word matching (where "swim" will not match "swimming"). This is 293 covered by the "Fuzzy Search" extension to IMAP [RFC6203], and that 294 extension is compatible with this one and can be combined with it. 296 Whether or not Fuzzy Search is implemented or used, this extension 297 explicitly allows flexible searching with respect to TEXT and BODY 298 searches. Servers MAY use fuzzy text matching in multimailbox 299 searches. 301 2.4. Server-Side Limitations on Search Volume 302 To avoid having a search use more than a reasonable share of server 303 resources, servers MAY apply limits that go beyond loop protection, 304 such as limits on the number of mailboxes that may be searched at 305 once, and/or limits on the number or total size of messages searched. 306 A server can apply those limits up front, responding with "NO 307 [LIMIT]" if a limit is exceeded (see [RFC5530] for information about 308 response codes). Alternatively, a server can process the search and 309 terminate it when a limit is exceeded, responding with "OK [LIMIT]" 310 and returning partial results. Note that searches that return 311 partial results can cause complexity for client implementations and 312 confusion to users. 314 3. Examples 316 In the following example, note that two ESEARCH commands are 317 pipelined, and that the server is running them in parallel, 318 interleaving a response to the second search amid the responses to 319 the first (watch the tags). 321 C: tag1 ESEARCH IN (mailboxes "folder1" subtree "folder2") unseen 322 C: tag2 ESEARCH IN (mailboxes "folder1" subtree-one "folder2") 323 subject "chad" 324 S: * ESEARCH (TAG "tag1" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 325 4001,4003,4005,4007,4009 326 S: * ESEARCH (TAG "tag2" MAILBOX "folder1" UIDVALIDITY 1) UID ALL 327 3001:3004,3788 328 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/banana" UIDVALIDITY 503) 329 UID ALL 3002,4004 330 S: * ESEARCH (TAG "tag1" MAILBOX "folder2/peach" UIDVALIDITY 3) UID 331 ALL 921691 332 S: tag1 OK done 333 S: * ESEARCH (TAG "tag2" MAILBOX "folder2/salmon" UIDVALIDITY 334 1111111) UID ALL 50003,50006,50009,50012 335 S: tag2 OK done 337 4. Formal Syntax 339 The following syntax specification uses the Augmented Backus-Naur 340 Form (ABNF) as described in [RFC5234]. Terms not defined here are 341 taken from [RFC3501], [RFC5465], or [RFC4466]. 343 command-auth =/ esearch 344 ; Update definition from IMAP base [RFC3501]. 345 ; Add new "esearch" command. 347 command-select =/ esearch 348 ; Update definition from IMAP base [RFC3501]. 349 ; Add new "esearch" command. 351 filter-mailboxes-other =/ ("subtree-one" SP one-or-more-mailbox) 352 ; Update definition from IMAP Notify [RFC5465]. 353 ; Add new "subtree-one" selector. 355 filter-mailboxes-selected = "selected" 356 ; Update definition from IMAP Notify [RFC5465]. 357 ; We forbid the use of "selected-delayed". 359 one-correlator = ("TAG" SP tag-string) / ("MAILBOX" SP astring) / 360 ("UIDVALIDITY" SP nz-number) 361 ; Each correlator MUST appear exactly once. 363 scope-option = scope-option-name [SP scope-option-value] 364 ; No options defined here. Syntax for future extensions. 366 scope-option-name = tagged-ext-label 367 ; No options defined here. Syntax for future extensions. 369 scope-option-value = tagged-ext-val 370 ; No options defined here. Syntax for future extensions. 372 scope-options = scope-option *(SP scope-option) 373 ; A given option may only appear once. 374 ; No options defined here. Syntax for future extensions. 376 esearch = "ESEARCH" [SP esearch-source-opts] 377 [SP search-return-opts] SP search-program 379 search-correlator = SP "(" one-correlator *(SP one-correlator) ")" 380 ; Updates definition in IMAP4 ABNF [RFC4466]. 382 esearch-source-opts = "IN" SP "(" source-mbox [SP 383 "(" scope-options ")"] ")" 385 source-mbox = filter-mailboxes *(SP filter-mailboxes) 386 ; "filter-mailboxes" is defined in IMAP Notify [RFC5465]. 387 ; See updated definition of filter-mailboxes-other, above. 388 ; See updated definition of filter-mailboxes-selected, above. 390 5. Security Considerations 392 This new IMAP ESEARCH command allows a single command to search many 393 mailboxes at once. On the one hand, a client could do that by 394 sending many IMAP SEARCH commands. On the other hand, this makes it 395 easier for a client to overwork a server, by sending a single command 396 that results in an expensive search of tens of thousands of 397 mailboxes. Server implementations need to be aware of that, and 398 provide mechanisms that prevent a client from adversely affecting 399 other users. Limitations on the number of mailboxes that may be 400 searched in one command, and/or on the server resources that will be 401 devoted to responding to a single client, are reasonable limitations 402 for an implementation to impose (see also Section 2.4. 404 Implementations MUST, of course, apply access controls appropriately, 405 limiting a user's access to ESEARCH in the same way its access is 406 limited for any other IMAP commands. This extension has no data- 407 access risks beyond what may be there in the unextended IMAP 408 implementation. 410 Mailboxes matching the source options for which the logged-in user 411 lacks sufficient rights MUST be ignored by the ESEARCH command 412 processing (see the paragraph about this in Section 2.2). In 413 particular, any attempt to distinguish insufficient access from non- 414 existent mailboxes may expose information about the mailbox hierarchy 415 that isn't otherwise available to the client. 417 If "subtree" is specified, the server MUST defend against loops in 418 the hierarchy (see the paragraph about this in Section 2.2). 420 6. IANA Considerations 422 The "IMAP Capabilities" registry is currently located at . 425 IANA is asked to change the reference for the IMAP capability 426 "MULTISEARCH" to point to this document. 428 7. Implementation Status 430 [[RFC Editor: Please remove this section at publication.]] 432 This section records the status of known implementations of the 433 protocol defined by this specification at the time of posting of 434 this Internet-Draft, and is based on a proposal described in RFC 435 6982. The description of implementations in this section is 436 intended to assist the IETF in its decision processes in 437 progressing drafts to RFCs. Please note that the listing of any 438 individual implementation here does not imply endorsement by the 439 IETF. Furthermore, no effort has been spent to verify the 440 information presented here that was supplied by IETF contributors. 441 This is not intended as, and must not be construed to be, a 442 catalog of available implementations or their features. Readers 443 are advised to note that other implementations may exist. 445 According to RFC 6982, "this will allow reviewers and working 446 groups to assign due consideration to documents that have the 447 benefit of running code, which may serve as evidence of valuable 448 experimentation and feedback that have made the implemented 449 protocols more mature. It is up to the individual working groups 450 to use this information as they see fit". 452 The following implementations are known to exist: 454 o Oracle has a server implementation that is not currently in a 455 product. 457 o There is a client implementation that has been tested with the 458 Oracle server. No further information is available. 460 Interest has been expressed in creating the following 461 implementations: 463 o Isode Limited 465 8. Changes Since RFC 6237 467 o Change to Standards Track. 469 o Added paragraph about duplicate mailboxes. 471 o Added Section 2.3 about fuzzy search. 473 o Added Section 7, "Implementation Status". 474 [[RFC Editor: Please remove this bullet at publication.]] 476 9. Acknowledgments 478 The authors gratefully acknowledge feedback provided by Timo 479 Sirainen, Peter Coates, Arnt Gulbrandsen, and Chris Newman. 481 10. References 483 10.1. Normative References 485 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 486 Requirement Levels", BCP 14, RFC 2119, March 1997. 488 [RFC2978] Freed, N. and J. Postel, "IANA Charset Registration 489 Procedures", BCP 19, RFC 2978, October 2000. 491 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 492 4rev1", RFC 3501, March 2003. 494 [RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", 495 RFC 4314, December 2005. 497 [RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 498 ABNF", RFC 4466, April 2006. 500 [RFC4731] Melnikov, A. and D. Cridland, "IMAP4 Extension to SEARCH 501 Command for Controlling What Kind of Information Is 502 Returned", RFC 4731, November 2006. 504 [RFC5182] Melnikov, A., "IMAP Extension for Referencing the Last 505 SEARCH Result", RFC 5182, March 2008. 507 [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax 508 Specifications: ABNF", STD 68, RFC 5234, January 2008. 510 [RFC5267] Cridland, D. and C. King, "Contexts for IMAP4", RFC 5267, 511 July 2008. 513 [RFC5465] Gulbrandsen, A., King, C. and A. Melnikov, "The IMAP 514 NOTIFY Extension", RFC 5465, February 2009. 516 [RFC5530] Gulbrandsen, A., "IMAP Response Codes", RFC 5530, May 517 2009. 519 10.2. Informative References 521 [RFC6203] Sirainen, T., "IMAP4 Extension for Fuzzy Search", RFC 522 6203, March 2011. 524 Authors' Addresses 526 Barry Leiba 527 Huawei Technologies 529 Phone: +1 646 827 0648 530 Email: barryleiba@computer.org 531 URI: http://internetmessagingtechnology.org/ 533 Alexey Melnikov 534 Isode Limited 535 5 Castle Business Village 536 36 Station Road 537 Hampton, Middlesex TW12 2BX 538 UK 540 Email: Alexey.Melnikov@isode.com 541 URI: http://www.melnikov.ca/