idnits 2.17.1 draft-ietf-lemonade-reconnect-client-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** It looks like you're using RFC 3978 boilerplate. You should update this to the boilerplate described in the IETF Trust License Policy document (see https://trustee.ietf.org/license-info), which is required now. -- Found old boilerplate from RFC 3978, Section 5.1 on line 17. -- Found old boilerplate from RFC 3978, Section 5.5 on line 1053. -- Found old boilerplate from RFC 3979, Section 5, paragraph 1 on line 1064. -- Found old boilerplate from RFC 3979, Section 5, paragraph 2 on line 1071. -- Found old boilerplate from RFC 3979, Section 5, paragraph 3 on line 1077. ** This document has an original RFC 3978 Section 5.4 Copyright Line, instead of the newer IETF Trust Copyright according to RFC 4748. ** This document has an original RFC 3978 Section 5.5 Disclaimer, instead of the newer disclaimer which includes the IETF Trust according to RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- No issues found here. Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- No issues found here. Miscellaneous warnings: ---------------------------------------------------------------------------- == The copyright year in the RFC 3978 Section 5.4 Copyright Line does not match the current year -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (November 24, 2006) is 6362 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) == Missing Reference: 'UIDVALIDITY 3857529045' is mentioned on line 874, but not defined == Missing Reference: 'UIDNEXT 550' is mentioned on line 230, but not defined == Missing Reference: 'HIGHESTMODSEQ 20060128194045007' is mentioned on line 231, but not defined == Missing Reference: 'UNSEEN 12' is mentioned on line 873, but not defined == Missing Reference: 'UIDVALIDITY 67890007' is mentioned on line 379, but not defined == Missing Reference: 'UIDNEXT 567' is mentioned on line 281, but not defined == Missing Reference: 'HIGHESTMODSEQ 20060115205545359' is mentioned on line 383, but not defined == Missing Reference: 'UNSEEN 7' is mentioned on line 385, but not defined == Missing Reference: 'UIDNEXT 30013' is mentioned on line 381, but not defined == Missing Reference: 'HIGHESTMODSEQ 20010715194045319' is mentioned on line 597, but not defined == Missing Reference: 'UIDNEXT 201' is mentioned on line 875, but not defined == Missing Reference: 'HIGHESTMODSEQ 20010715194045007' is mentioned on line 878, but not defined ** Obsolete normative reference: RFC 4234 (ref. 'ABNF') (Obsoleted by RFC 5234) ** Obsolete normative reference: RFC 4551 (ref. 'CONDSTORE') (Obsoleted by RFC 7162) ** Obsolete normative reference: RFC 3501 (Obsoleted by RFC 9051) Summary: 6 errors (**), 0 flaws (~~), 13 warnings (==), 7 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 Network Working Group A. Melnikov 3 Internet-Draft D. Cridland 4 Intended status: Standards Track Isode Ltd 5 Expires: May 28, 2007 C. Wilson 6 Nokia 7 November 24, 2006 9 IMAP4 Extensions for Quick Mailbox Resynchronization 10 draft-ietf-lemonade-reconnect-client-02.txt 12 Status of this Memo 14 By submitting this Internet-Draft, each author represents that any 15 applicable patent or other IPR claims of which he or she is aware 16 have been or will be disclosed, and any of which he or she becomes 17 aware will be disclosed, in accordance with Section 6 of BCP 79. 19 Internet-Drafts are working documents of the Internet Engineering 20 Task Force (IETF), its areas, and its working groups. Note that 21 other groups may also distribute working documents as Internet- 22 Drafts. 24 Internet-Drafts are draft documents valid for a maximum of six months 25 and may be updated, replaced, or obsoleted by other documents at any 26 time. It is inappropriate to use Internet-Drafts as reference 27 material or to cite them other than as "work in progress." 29 The list of current Internet-Drafts can be accessed at 30 http://www.ietf.org/ietf/1id-abstracts.txt. 32 The list of Internet-Draft Shadow Directories can be accessed at 33 http://www.ietf.org/shadow.html. 35 This Internet-Draft will expire on May 28, 2007. 37 Copyright Notice 39 Copyright (C) The Internet Society (2006). 41 Abstract 43 This document defines an IMAP4 extension, which gives an IMAP client 44 the ability to quickly resynchronize any previously opened mailbox as 45 part of the SELECT command, without the need for server-side state or 46 additional client round-trips. This extension also introduces a new 47 response that allows for a more compact representation for a list of 48 expunged messages. 50 Changes since draft-ietf-lemonade-reconnect-client-01.txt 52 o Folded the EXPUNGED extension 53 (draft-melnikov-imap-expunged-02.txt) into this document. Updated 54 mailbox synchronization instructions. 56 o Added UID sequence number matching. 58 o Clarified how NOMODSEQ response affects this extension. 60 o Other minor editorial changes and fixes. 62 Changes since draft-ietf-lemonade-reconnect-client-00.txt 64 o Changed server behavior when the specified UIDVALIDITY doesn't 65 match the current. This allows the client to chose how to proceed 66 after that. 68 o If client's UIDVALIDITY doesn't match server's, the server will 69 not return any flags anymore. 71 o Clarified that SELECT (QRESYNC) is a CONDSTORE-enabling command. 73 o Other minor editorial changes and fixes. 75 Table of Contents 77 1. Requirements notation . . . . . . . . . . . . . . . . . . . . 4 78 2. Introduction and Overview . . . . . . . . . . . . . . . . . . 4 79 3. IMAP Protocol Changes . . . . . . . . . . . . . . . . . . . . 5 80 3.1. QRESYNC parameter to SELECT/EXAMINE . . . . . . . . . . . 5 81 3.2. VANISHED UID FETCH modifier . . . . . . . . . . . . . . . 10 82 3.3. EXPUNGE Command . . . . . . . . . . . . . . . . . . . . . 11 83 3.4. CLOSE Command . . . . . . . . . . . . . . . . . . . . . . 12 84 3.5. UID EXPUNGE Command . . . . . . . . . . . . . . . . . . . 13 85 3.6. VANISHED Response . . . . . . . . . . . . . . . . . . . . 14 86 4. Server implementation considerations . . . . . . . . . . . . . 17 87 4.1. Server implementations that don't store extra state . . . 17 88 4.2. Server implementations storing minimal state . . . . . . . 17 89 4.3. Additional state required on the server . . . . . . . . . 17 90 5. Updated synchronization sequence . . . . . . . . . . . . . . . 19 91 6. Formal Syntax . . . . . . . . . . . . . . . . . . . . . . . . 21 92 7. Security Considerations . . . . . . . . . . . . . . . . . . . 22 93 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 94 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 23 95 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 96 10.1. Normative References . . . . . . . . . . . . . . . . . . . 23 97 10.2. Informative References . . . . . . . . . . . . . . . . . . 24 98 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 24 99 Intellectual Property and Copyright Statements . . . . . . . . . . 25 101 1. Requirements notation 103 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", 104 "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this 105 document are to be interpreted as described in [RFC2119]. 107 In examples, "C:" and "S:" indicate lines sent by the client and 108 server respectively. If a single "C:" or "S:" label applies to 109 multiple lines, then the line breaks between those lines are for 110 editorial clarity only and are not part of the actual protocol 111 exchange. 113 Understanding of the IMAP message sequence numbers and UIDs and the 114 EXPUNGE response [RFC3501] is essential when reading this document. 116 [[anchor2: Editorial comments and questions are marked like this.]] 118 2. Introduction and Overview 120 The [CONDSTORE] extension gives a disconnected client ability to 121 quickly resynchronize IMAP flag changes for previously seen messages. 122 This can be done using the CHANGEDSINCE FETCH modifier once a mailbox 123 is opened. In order for the client to discover which messages have 124 been expunged, the client still has to issue a UID FETCH or a UID 125 SEARCH command. This document defines an extension to [CONDSTORE] 126 that allows a reconnecting client to perform full resynchronization, 127 including discovery of expunged messages, in a single round-trip. 128 This extension also introduces a new response VANISHED that allows 129 for a more compact representation for a list of expunged messages. 131 This extension can be useful for mobile clients that can experience 132 frequent disconnects caused by environmental factors (battery life, 133 signal strength, etc.). Such clients would need a way to quickly 134 reconnect to the IMAP server, without forcing the user to experience 135 long delay and pay big bills for the amount of traffic generated by 136 resynchronization. 138 By extending the SELECT command to perform the additional 139 resynchronization, this also allows clients to reduce concurrent 140 connections to the IMAP server held purely for the sake of avoiding 141 the resynchronization. 143 [[anchor4: Note to RFC editor: Please change the capability name 144 everywhere to "QRESYNC".]] 146 The quick resync IMAP extension is present if an IMAP4 server returns 147 "X-DRAFT-W02-QRESYNC" as one of the supported capabilities to the 148 CAPABILITY command. Note, that this extension REQUIREs support for 149 the [CONDSTORE] IMAP extension, so it MUST be announced in the 150 CAPABILITY response as well. 152 This document puts additional requirements on a server implementing 153 the [CONDSTORE] extension. Each mailbox that supports persistent 154 storage of mod-sequences, i.e., for which the server has sent a 155 HIGHESTMODSEQ untagged OK response code on a successful SELECT/ 156 EXAMINE, MUST increment the per-mailbox mod-sequence when one or more 157 messages are expunged due to EXPUNGE, UID EXPUNGE or CLOSE; the 158 server MUST associate the incremented mod-sequence with the UIDs of 159 the expunged messages. 161 A client which supports CONDSTORE but not this extension might 162 resynchronize a mailbox and discover that its HIGHESTMODSEQ has 163 increased from the value cached by the client. If the increase is 164 due only to messages having been expunged since the client last 165 synchronized, the client is likely to send a FETCH ... CHANGEDSINCE 166 command that returns no data. Thus, a client which supports 167 CONDSTORE but not this extension might incur a penalty of an unneeded 168 round-trip when resynchronizing some mailboxes (those which have had 169 messages expunged but no flag changes since the last 170 synchronization). 172 This extra round-trip is only incurred by clients that supports 173 CONDSTORE but not this extension, and only when a mailbox has had 174 messages expunged but no flag changes to non-expunged messages. 175 Since CONDSTORE is a relatively new extension, it is thought likely 176 that clients that support it will also support this extension. 178 3. IMAP Protocol Changes 180 3.1. QRESYNC parameter to SELECT/EXAMINE 182 The Quick Resynchronization parameter to SELECT/EXAMINE commands has 183 four arguments: 185 o the last known UIDVALIDITY, 187 o the last known modification sequence 189 o the optional set of known UIDs 191 o an optional parenthesized list of known sequence ranges and their 192 corresponding UIDs. 194 The parameter acts as a CONDSTORE enabling command, as defined in 196 [CONDSTORE]. In other words, the use of the QRESYNC parameter 197 implies the CONDSTORE parameter. The QRESYNC parameter also tells 198 the server that it SHOULD start sending VANISHED responses (see 199 Section 3.6) instead of EXPUNGE responses. This change remains in 200 effect until the connection is closed. 202 Before opening the specified mailbox the server verifies all 203 arguments for syntactic validity. If any parameter is not 204 syntactically valid, the server returns the tagged BAD response, and 205 the mailbox remains unselected. Once the check is done the server 206 opens the mailbox as if no SELECT/EXAMINE parameters are specified 207 (this is subject to processing of other parameters as defined in 208 other extensions). In particular this means that server MUST send 209 all untagged responses as specified in Section 6.3.1/6.3.2 of 210 [RFC3501]. 212 After that the server checks the UIDVALIDITY value provided by the 213 client. If the provided UIDVALIDITY doesn't match the UIDVALIDITY 214 for the mailbox being opened, then the server MUST ignore the 215 remaining parameters and behave as if no dynamic message data 216 changed. The client can discover this situation by comparing the 217 UIDVALIDITY value returned by the server. This behaviour allows the 218 client not to synchronize the mailbox or decide on the best 219 synchronization strategy. 221 Example: Attempting to resynchronize INBOX, but the provided 222 UIDVALIDITY parameter doesn't match the current UIDVALIDITY 223 value. 225 C: A02 SELECT INBOX (QRESYNC (67890007 20050715194045000 226 41,43:211,214:541)) 227 S: * 464 EXISTS 228 S: * 3 RECENT 229 S: * OK [UIDVALIDITY 3857529045] UIDVALIDITY 230 S: * OK [UIDNEXT 550] Predicted next UID 231 S: * OK [HIGHESTMODSEQ 20060128194045007] 232 S: * OK [UNSEEN 12] Message 12 is first unseen 233 S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) 234 S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft 235 \Deleted \Seen \*)] Permanent flags 236 S: A02 OK [READ-WRITE] Sorry, UIDVALIDITY mismatch 238 Modification Sequence and UID Parameters: 240 A server that doesn't support the persistent storage of mod-sequences 241 for the mailbox MUST send the OK untagged response including the 242 NOMODSEQ response code with every successful SELECT or EXAMINE 243 command, as described in [CONDSTORE]. Such server doesn't need to 244 remember mod-sequences for expunged messages in the mailbox. It MUST 245 ignore the remaining parameters and behave as if no dynamic message 246 data changed. 248 However, whether the server returns the HIGHESTMODSEQ or the NOMODSEQ 249 response code, the QRESYNC parameter still enables the use of the 250 VANISHED response in lieu of the EXPUNGE response Section 3.6. 252 If the provided UIDVALIDITY matches that of the selected mailbox, the 253 server then checks the last known modification sequence. 254 The server sends the client any pending flag changes (using FETCH 255 responses that MUST contain UIDs) and expunges that have occurred in 256 this mailbox since the provided modification sequence. 258 If the list of known UIDs was also provided, the server should only 259 report flag changes and expunges for the provided messages. If the 260 client did not provide the list of UIDs, the server acts as if the 261 client has specified "1:*". 263 Thus, the client can process just these pending events and need not 264 perform a full resynchronization. Without the message sequence 265 number matching information, the result of this step is semantically 266 equivalent to the client issuing: 267 tag1 UID FETCH "known-uids" (FLAGS) (CHANGEDSINCE 268 "mod-sequence-value" VANISHED) 270 Example: 272 C: A02 SELECT INBOX (QRESYNC (67890007 273 20060115194045000 41,43:211,214:541)) 275 S: * 314 EXISTS 277 S: 15 RECENT 279 S: * OK [UIDVALIDITY 67890007] UIDVALIDITY 281 S: * OK [UIDNEXT 567] Predicted next UID 283 S: * OK [HIGHESTMODSEQ 20060115205545359] 285 S: * OK [UNSEEN 7] There are some unseen messages in the mailbox 287 S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) 288 S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft 289 \Deleted \Seen \*)] Permanent flags 291 S: * 49 FETCH (UID 117 FLAGS (\Seen \Answered)) 293 S: * 50 FETCH (UID 119 FLAGS (\Draft $MDNSent)) 295 S: ... 297 S: * 100 FETCH (UID 541 FLAGS (\Seen $Forwarded)) 299 S: * VANISHED 41,43:116,118,120:211,214:540 301 S: A02 OK [READ-WRITE] mailbox selected 303 Message sequence match data: 305 A client MAY provide a parenthesized list of a message sequence set 306 and the corresponding UID sets. Both MUST be provided in ascending 307 order. The server uses this data to restrict the range for which it 308 provides expunged message information. 310 Conceptually, the client provides a small sample of sequence numbers 311 for which it knows the corresponding UIDs. The server then compares 312 each sequence number and UID pair the client provides with the 313 current state of the mailbox. If a pair matches, then the client 314 knows of any expunges up to, and including, the message, and thus 315 will not include that range in the VANISHED response, even if the 316 "mod-sequence-value" provided by the client is too old for the server 317 to have data of when those messages were expunged. 319 Thus if the Nth message number in the first set in the list is 4, and 320 the Nth UID in the second set in the list is 8, and the mailbox's 321 fourth message has UID 8, then no UIDs equal to or less than 8 are 322 present in the VANISHED response. If the (N+1)th message number is 323 12, and the (N+1)th UID is 24, and the (N+1)th message in the mailbox 324 has UID 25, then the lowest UID included in the VANISHED response 325 would be 9. 327 In the following two examples, the server is unable to remember 328 expunges at all, and only UIDs with messages divisible by three are 329 present in the mailbox. In the first example, the client does not 330 use the fourth parameter, in the second, it provides it. This 331 example is somewhat extreme, but shows that judicious usage of the 332 sequence match data can save a substantial amount of bandwidth. 334 Example: 336 C: A02 SELECT INBOX (QRESYNC (67890007 337 20060115194045000 1:29997)) 339 S: * 10003 EXISTS 341 S: 5 RECENT 343 S: * OK [UIDVALIDITY 67890007] UIDVALIDITY 345 S: * OK [UIDNEXT 30013] Predicted next UID 347 S: * OK [HIGHESTMODSEQ 20060115205545359] 349 S: * OK [UNSEEN 7] There are some unseen messages in the mailbox 351 S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) 353 S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft 354 \Deleted \Seen \*)] Permanent flags 356 S: * 9889 FETCH (UID 29667 FLAGS (\Seen \Answered)) 358 S: * 9890 FETCH (UID 29670 FLAGS (\Draft $MDNSent)) 360 S: ... 362 S: * 9999 FETCH (UID 29997 FLAGS (\Seen $Forwarded)) 364 S: * VANISHED 1:2,4:5,7:8,10:11,13:14 [...] 365 29998:29999,30001:30002,30004:30005,30007:30008 367 S: A02 OK [READ-WRITE] mailbox selected 369 Example: 371 C: A02 SELECT INBOX (QRESYNC (67890007 372 20060115194045000 1:29997 (5000,7500,9000,9990:9999 15000, 373 22500,27000,29970,29973,29976,29979,29982,29985,29988,29991, 374 29994,29997)) 376 S: * 10003 EXISTS 378 S: 5 RECENT 379 S: * OK [UIDVALIDITY 67890007] UIDVALIDITY 381 S: * OK [UIDNEXT 30013] Predicted next UID 383 S: * OK [HIGHESTMODSEQ 20060115205545359] 385 S: * OK [UNSEEN 7] There are some unseen messages in the mailbox 387 S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen) 389 S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft 390 \Deleted \Seen \*)] Permanent flags 392 S: * 9889 FETCH (UID 29667 FLAGS (\Seen \Answered)) 394 S: * 9890 FETCH (UID 29670 FLAGS (\Draft $MDNSent)) 396 S: ... 398 S: * 9999 FETCH (UID 29997 FLAGS (\Seen $Forwarded)) 400 S: * VANISHED 29998:29999,30001:30002,30004:30005,30007:30008 402 S: A02 OK [READ-WRITE] mailbox selected 404 3.2. VANISHED UID FETCH modifier 406 [IMAPABNF] has extended the syntax of the FETCH and UID FETCH 407 commands to include an optional FETCH modifier. This document 408 defines a new UID FETCH modifier: VANISHED. 410 Note, that the VANISHED UID FETCH modifier is NOT allowed with a 411 FETCH command. The server MUST return a tagged BAD response if this 412 response is specified as a modifier to the FETCH command. 414 The VANISHED UID FETCH modifier MUST only be specified together with 415 the CHANGEDSINCE UID FETCH modifier. 417 The VANISHED UID FETCH modifier instructs the server to report those 418 messages from the UID set parameter that have been expunged and whose 419 associated modsequence is larger than the specified modsequence. 420 That is, the client requests to be informed of messages from the 421 specified set that were expunged since the specified modsequence. 422 Note that the modsequence(s) associated with these messages were 423 updated when the messages were expunged (as described above). The 424 expunged messages are reported using the VANISHED response as 425 described in Section 3.6, which MUST contain the TAG correlator. 427 Note: a server that receives a modsequence smaller than any of the 428 expunged modsequence it remembers about MUST behave as if it was 429 requested to report all expunged messages from the provided UID set 430 parameter. 432 The VANISHED UID FETCH modifier also instructs the server to replace 433 all further EXPUNGE responses with VANISHED responses. The server 434 MUST do this until the connection is closed. 436 Example 1: Without the VANISHED UID FETCH modifier a CONDSTORE-aware 437 client [CONDSTORE] needs to issue separate commands to learn of flag 438 changes and expunged messages since the last synchronization: 440 C: s100 UID FETCH 300:500 (FLAGS) (CHANGEDSINCE 12345) 441 S: * 1 FETCH (UID 404 MODSEQ (65402) FLAGS (\Seen)) 442 S: * 2 FETCH (UID 406 MODSEQ (75403) FLAGS (\Deleted)) 443 S: * 4 FETCH (UID 408 MODSEQ (29738) FLAGS ($NoJunk 444 $AutoJunk $MDNSent)) 445 S: s100 OK FETCH completed 446 C: s101 UID SEARCH 300:500 447 S: * SEARCH 404 406 407 408 410 412 448 S: s101 OK search completed 450 Where 300 and 500 are the lowest and highest UIDs from client's 451 cache. The second SEARCH response tells the client that the messages 452 with UIDs 407, 410 and 412 are still present, but their flags haven't 453 changed since the specified modification sequence. 455 Using the VANISHED UID FETCH modifier it is sufficient to issue only 456 a single command: 458 C: s100 UID FETCH 300:500 (FLAGS) (CHANGEDSINCE 12345 459 VANISHED) 460 S: * 1 FETCH (UID 404 MODSEQ (65402) FLAGS (\Seen)) 461 S: * 2 FETCH (UID 406 MODSEQ (75403) FLAGS (\Deleted)) 462 S: * 4 FETCH (UID 408 MODSEQ (29738) FLAGS ($NoJunk 463 $AutoJunk $MDNSent)) 464 S: * VANISHED (TAG "s100") 300:310,405,411 465 S: s100 OK FETCH completed 467 3.3. EXPUNGE Command 469 Arguments: none 471 Responses: untagged responses: EXPUNGE or VANISHED 472 Result: OK - expunge completed 473 NO - expunge failure: can't expunge (e.g., permission 474 denied) 475 BAD - command unknown or arguments invalid 477 This section updates the definition of the EXPUNGE command described 478 in section 6.4.3 of [RFC3501]. 480 The EXPUNGE command permanently removes all messages that have the 481 \Deleted flag set from the currently selected mailbox. Before 482 returning an OK to the client, those messages that are removed are 483 reported using a VANISHED response or EXPUNGE responses. 485 If the server is capable of storing modification sequences for the 486 selected mailbox, it MUST increment the per-mailbox mod-sequence if 487 at least one message was permanently removed due to the execution of 488 the EXPUNGE command. For each permanently removed message the server 489 MUST remember the incremented mod-sequence and corresponding UID. If 490 at least one message got expunged, the server MUST send the updated 491 per-mailbox modification sequence using the HIGHESTMODSEQ response 492 code (defined in [CONDSTORE]) in the tagged OK response. 494 Example: C: A202 EXPUNGE 495 S: * 3 EXPUNGE 496 S: * 3 EXPUNGE 497 S: * 5 EXPUNGE 498 S: * 8 EXPUNGE 499 S: A202 OK [HIGHESTMODSEQ 20010715194045319] expunged 501 Note: In this example, messages 3, 4, 7, and 11 had the \Deleted flag 502 set. See the description of the EXPUNGE response in [RFC3501] for 503 further explanation. 505 Note that once the VANISHED response is enabled on the connection the 506 previous example might look like this: 508 Example: C: B202 EXPUNGE 509 S: * VANISHED 405,407,410,425 510 S: B202 OK [HIGHESTMODSEQ 20010715194045319] expunged 512 Here messages with message numbers 3, 4, 7 and 11 have respective 513 UIDs 405, 407, 410 and 425. 515 3.4. CLOSE Command 516 Arguments: none 518 Responses: no specific responses for this command 520 Result: OK - close completed, now in authenticated state 521 BAD - command unknown or arguments invalid 523 This section updates the definition of the CLOSE command described in 524 section 6.4.2 of [RFC3501]. 526 The CLOSE command permanently removes all messages that have the 527 \Deleted flag set from the currently selected mailbox, and returns to 528 the authenticated state from the selected state. No untagged EXPUNGE 529 (or VANISHED) responses are sent. 531 If the server is capable of storing modification sequences for the 532 selected mailbox, it MUST increment the per-mailbox mod-sequence if 533 at least one message was permanently removed due to the execution of 534 the CLOSE command. For each permanently removed message the server 535 MUST remember the incremented mod-sequence and corresponding UID. If 536 at least one message got expunged, the server MUST send the updated 537 per-mailbox modification sequence using the HIGHESTMODSEQ response 538 code (defined in [CONDSTORE]) in the tagged OK response. 540 Example: C: A202 CLOSE 541 S: A202 OK [HIGHESTMODSEQ 20010715194045319] done 543 3.5. UID EXPUNGE Command 545 Arguments: message set 547 Responses: untagged responses: EXPUNGE or VANISHED 549 Result: OK - expunge completed 550 NO - expunge failure: can't expunge (e.g., permission 551 denied) 552 BAD - command unknown or arguments invalid 554 This section updates the definition of the UID EXPUNGE command 555 described in section 2.1 of [UIDPLUS]. Servers that implement both 556 [UIDPLUS] and X-DRAFT-I02-EXPUNGED extensions must implement UID 557 EXPUNGE as described in this section. 559 The UID EXPUNGE command permanently removes from the currently 560 selected mailbox all messages that both have the \Deleted flag set 561 and have a UID that is included in the specified message set. If a 562 message either does not have the \Deleted flag set or has a UID that 563 is not included in the specified message set, it is not affected. 565 This command is particularly useful for disconnected mode clients. 566 By using UID EXPUNGE instead of EXPUNGE when resynchronizing with the 567 server, the client can avoid inadvertently removing any messages that 568 have been marked as \Deleted by other clients between the time that 569 the client was last connected and the time the client resynchronizes. 571 If the server does not support the UIDPLUS capability, the client 572 SHOULD fall back to using the STORE command to temporarily remove the 573 \Deleted flag from messages it does not want to remove, then issuing 574 the EXPUNGE command. Finally, the client SHOULD use the STORE 575 command to restore the \Deleted flag on the messages in which it was 576 temporarily removed. 578 Alternatively, the client MAY fall back to using just the EXPUNGE 579 command, risking the unintended removal of some messages. 581 Before returning an OK to the client, those messages that are removed 582 are reported using a VANISHED response or EXPUNGE responses. 584 If the server is capable of storing modification sequences for the 585 selected mailbox, it MUST increment the per-mailbox mod-sequence if 586 at least one message was permanently removed due to the execution of 587 the UID EXPUNGE command. For each permanently removed message the 588 server MUST remember the incremented mod-sequence and corresponding 589 UID. If at least one message got expunged, the server MUST send the 590 updated per-mailbox modification sequence using the HIGHESTMODSEQ 591 response code (defined in [CONDSTORE]) in the tagged OK response. 593 Example: C: . UID EXPUNGE 3000:3002 594 S: * 3 EXPUNGE 595 S: * 3 EXPUNGE 596 S: * 3 EXPUNGE 597 S: . OK [HIGHESTMODSEQ 20010715194045319] Ok 599 Note: In this example, at least messages with message numbers 3, 4, 600 and 5 (UIDs 3000 to 3002) had the \Deleted flag set. See the 601 description of the EXPUNGE response in [RFC3501] for further 602 explanation. 604 3.6. VANISHED Response 605 Contents: optional correlators 607 list of UIDs 609 The VANISHED response reports that the specified UIDs have been 610 permanently removed from the mailbox. This response is similar to 611 the EXPUNGE response [RFC3501], however it can return information 612 about multiple messages and it returns UIDs instead of message 613 numbers. The first benefit saves bandwidth, while the second is more 614 convenient for clients which only use UIDs to access the IMAP server. 616 The VANISHED response has the same restrictions on when it can be 617 sent as does the EXPUNGE response (see below). 619 The VANISHED response starts with an optional correlator. If it is 620 present and contains the TAG correlator type, then the response is a 621 result of a UID FETCH (VANISHED) or a SELECT/EXAMINE (QRESYNC) 622 command. Other correlators can be added in the future. 624 The VANISHED response is sent as a result of a UID FETCH (VANISHED) 625 command, if the UID set parameter to the UID FETCH (VANISHED) command 626 includes UIDs of messages that are no longer in the mailbox. Such 627 VANISHED response MUST contain the TAG correlator. 629 Once a client has used "(VANISHED)" with a UID FETCH or "(QRESYNC)" 630 with SELECT/EXAMINE command, the server SHOULD use the VANISHED 631 response instead of the EXPUNGE response. The server SHOULD continue 632 using VANISHED in lieu of EXPUNGE for the duration of the connection. 633 In particular this affects the EXPUNGE [RFC3501] and UID EXPUNGE 634 [UIDPLUS] commands, as well as messages expunged in other 635 connections. Such VANISHED response MUST NOT contain the TAG 636 correlator. 638 A VANISHED response sent because of an EXPUNGE or UID EXPUNGE command 639 or because messages were expunged in other connections also 640 decrements the number of messages in the mailbox; it is not necessary 641 for the server to send an EXISTS and/or RECENT response with the new 642 value. It also decrements message sequence numbers for each 643 successive message in the mailbox (see the example at the end of this 644 section). Note that a VANISHED response caused by EXPUNGE/UID 645 EXPUNGE/messages expunged in other connections SHOULD only contain 646 UIDs for messages expunged since the last VANISHED/EXPUNGE response 647 sent for the currently opened mailbox or since the mailbox was 648 opened. That is, servers SHOULD NOT send UIDs for previously 649 expunged messages, unless explicitly requested to do so by the UID 650 FETCH (VANISHED) command. 652 Note that client implementors must take care to properly decrement 653 the number of messages in the mailbox even if a server violates this 654 last SHOULD or repeats the same UID multiple times in the returned 655 UID set. In general this means that a client using this extension 656 should either avoid using message numbers entirely, or have a 657 complete map of UID-to-message mapping for the selected mailbox. 659 A VANISHED response MUST NOT be sent when no command is in progress, 660 nor while responding to a FETCH, STORE, or SEARCH command. This rule 661 is necessary to prevent a loss of synchronization of message sequence 662 numbers between client and server. A command is not "in progress" 663 until the complete command has been received; in particular, a 664 command is not "in progress" during the negotiation of command 665 continuation. 667 Note: UID FETCH, UID STORE, and UID SEARCH are different commands 668 from FETCH, STORE, and SEARCH. A VANISHED response MAY be sent 669 during a UID command. However, the VANISHED response MUST NOT be 670 sent during a UID SEARCH command that contains message numbers in the 671 search criteria. 673 The update from the VANISHED response MUST be recorded by the client. 675 Example: Let's assume that there is the following mapping between 676 message numbers and UIDs in the currently selected mailbox (here "X" 677 marks messages with the \Deleted flag set, and "x" represents UIDs 678 which are not relevant for the example): 680 Message numbers: 1 2 3 4 5 6 7 8 9 10 11 681 UIDs: x 504 505 507 508 x 510 x x x 625 682 \Deleted messages: X X X X 684 In the presence of the extension defined in this document: 686 C: A202 EXPUNGE 687 S: * VANISHED 505,507,510,625 688 S: A202 OK EXPUNGE completed 690 Without the X-DRAFT-W02-QRESYNC [[anchor11: change before 691 publication]] extension the same example might look like: 693 C: A202 EXPUNGE 694 S: * 3 EXPUNGE 695 S: * 3 EXPUNGE 696 S: * 5 EXPUNGE 697 S: * 8 EXPUNGE 698 S: A202 OK EXPUNGE completed 699 (Continuing previous example) If subsequently messages with UIDs 504 700 and 508 got marked as \Deleted: 702 C: A210 EXPUNGE 703 S: * VANISHED 504,508 704 S: A210 OK EXPUNGE completed 706 I.e., the last VANISHED response only contains UIDs of messages 707 expunged since the previous VANISHED response. 709 4. Server implementation considerations 711 This section describes a minimalist implementation, a moderate 712 implementation, and an example of a full implementation. 714 4.1. Server implementations that don't store extra state 716 Strictly speaking, a server implementation that doesn't remember 717 modsequences associated with expunged messages can be considered 718 compliant with this specification. Such implementations return all 719 expunged messages specified in the UID set of the UID FETCH 720 (VANISHED) command every time, without paying attention to the 721 specified CHANGEDSINCE modsequence. Such implementations are 722 discouraged, as they can end up returning VANISHED responses bigger 723 than the result of a UID SEARCH command for the same UID set. 725 Clients which use the message sequence match data can reduce the 726 scope of this VANISHED response substantially in the typical case 727 where expunges have not happened, or happen only toward the end of 728 the mailbox. 730 4.2. Server implementations storing minimal state 732 A server which stores the HIGHESTMODSEQ value at the time of the last 733 EXPUNGE can omit the VANISHED response when a client provides a 734 MODSEQ value that is equal to, or higher than, the current value of 735 this datum - that is, when there have been no EXPUNGEs 737 A client providing message sequence match data can reduce the scope 738 as above. In the case where there have been no expunges, the server 739 can ignore this data. 741 4.3. Additional state required on the server 743 When compared to the [CONDSTORE] extension, this extension requires 744 servers to store additional state associated with expunged messages. 745 Note that implementations are not required to store this state in 746 persistent storage, however use of persistent storage is advisable. 748 One possible way to correctly implement the extension described in 749 this document would be to store a queue of 750 pairs. can be represented as a sequence of pairs. 753 When messages are expunged, one or more entry is added to the queue 754 tail. 756 When the server receives a request to return expunged messages since 757 a given modsequence, it will search the queue from the tail (i.e. 758 going from the highest expunged modsequence to the lowest), until it 759 sees the first record with a modsequence less than or equal to the 760 given modsequence, or it reaches the head of the queue. 762 Note that indefinitely storing information about expunged messages 763 can cause storage and related problems for an implementation. In the 764 worst case, this could result in almost 64Gb of storage for each IMAP 765 mailbox. For example, consider an implementation that stores triples for each range of messages 767 expunged at the same time. Each triple requires 16 octets: 4 octets 768 for each of the two UIDs, and 8 octets for the modsequence. Assume a 769 mailbox containing a single message with a UID of 2**32-1 (the 770 maximum possible UID value), where messages had previously existed 771 with UIDs starting at 1, and have been expunged one at a time. For 772 this mailbox alone, storage is required for the triples <1, 1, 773 modseq1>, <2, 2, modseq2>, ..., <2**32-2, 2**32-2, modseq4294967294>. 775 Hence, implementations are encouraged to adopt strategies to protect 776 against such storage problems, such as limiting the size of the queue 777 used to store modsequences for expunged messages and "expiring" older 778 records when this limit is reached. When the selected 779 implementation-specific queue limit is reached, the oldest record(s) 780 are deleted from the queue (Note that such records are located at the 781 queue head). For all such "expired" records the server needs to 782 store a single modsequence, which is the highest modsequence for all 783 "expired" expunged messages. 785 Note that if the client provides the message sequence match data, 786 this can heavily reduce the data cost of sending a complete set of 787 missing UIDs, thus reducing the problems for clients if a server is 788 unable to persist much of this queue. If the queue contains data 789 back to the requested modsequence, this data can be ignored. 791 Also note that if the UIDVALIDITY of the mailbox changes or if the 792 mailbox is deleted, then any state associated with expunged messages 793 MUST be deleted as well. 795 5. Updated synchronization sequence 797 This section updates the description of optimized synchronization in 798 section 6.1 of the [IMAP-DISC]. 800 An advanced disconnected mail client should use the X-DRAFT-W02- 801 QRESYNC [[anchor17: Fix before publication]] and [CONDSTORE] 802 extensions when they are supported by the server. The client MUST 803 cache the value from HIGHESTMODSEQ OK response code received on 804 mailbox opening and update it whenever the server sends MODSEQ FETCH 805 data items or HIGHESTMODSEQ response code. 807 When opening the mailbox for synchronization the client uses QRESYNC 808 parameter to the SELECT/EXAMINE command. The QRESYNC parameter is 809 followed by the UIDVALIDITY and mailbox HIGHESTMODSEQ values, as 810 known to the client. It can be optionally followed by the set of 811 UIDs, for example if the client is only interested in partial 812 synchronization of the mailbox. The client may also transmit a list 813 containing its knowledge of message numbers. 815 If the SELECT/EXAMINE command is successful, the client compares 816 UIDVALIDITY as described in step d)1) in section 3 of the 817 [IMAP-DISC]. If the cached UIDVALIDITY value matches the one 818 returned by the server and the server also returns the HIGHESTMODSEQ 819 response code, then the server reports expunged messages/returns flag 820 changes for all messages specified by the client in the UID set 821 parameter (or for all messages in the mailbox, if the client omitted 822 the UID set parameter). At this point the client is synchronized, 823 except for maybe the new messages. 825 If upon a successful SELECT/EXAMINE (QRESYNC) command the client 826 receives a NOMODSEQ OK untagged response (instead of the 827 HIGHESTMODSEQ response code), it MUST remove the last known 828 HIGHESTMODSEQ value from its cache and follow the more general 829 instructions in section 3 of the [IMAP-DISC]. 831 At this point the client is in sync with the server regarding old 832 messages. This client can now fetch information about new messages 833 (if requested by the user). 835 Step d) ("Server-to-client synchronization") in section 4 of the 836 [IMAP-DISC] in the presence of the X-DRAFT-W02-QRESYNC & CONDSTORE 837 extensions is amended as follows: 839 d) "Server-to-client synchronization" -- for each mailbox that 840 requires synchronization, do the following: 842 1a) Check the mailbox UIDVALIDITY (see section 4.1 of the 843 [IMAP-DISC] for more details) after issuing SELECT/ 844 EXAMINE (QRESYNC) command. If the UIDVALIDITY value 845 returned by the server differs, the client MUST 847 * empty the local cache of that mailbox; 849 * "forget" the cached HIGHESTMODSEQ value for 850 the mailbox; 852 * remove any pending "actions" which refer to 853 UIDs in that mailbox. Note, this doesn't 854 affect actions performed on client generated 855 fake UIDs (see section 5 of the [IMAP-DISC]); 857 * skip steps 1b and 2-II; 859 2) Fetch the current "descriptors"; 861 I) Discover new messages. 863 3) Fetch the bodies of any "interesting" messages that the 864 client doesn't already have. 866 Example: The UIDVALIDITY value is the same, but the HIGHESTMODSEQ 867 value has changed on the server while the client was 868 offline: 870 C: A142 SELECT INBOX (QRESYNC (3857529045 20010715194032001 1:198)) 871 S: * 172 EXISTS 872 S: * 1 RECENT 873 S: * OK [UNSEEN 12] Message 12 is first unseen 874 S: * OK [UIDVALIDITY 3857529045] UIDs valid 875 S: * OK [UIDNEXT 201] Predicted next UID 876 S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) 877 S: * OK [PERMANENTFLAGS (\Deleted \Seen \*)] Limited 878 S: * OK [HIGHESTMODSEQ 20010715194045007] 879 S: * 2 FETCH (UID 6 MODSEQ (20010715205008000) 880 FLAGS (\Deleted)) 881 S: * 5 FETCH (UID 9 MODSEQ (20010715195517000) 882 FLAGS ($NoJunk $AutoJunk $MDNSent)) 883 ... 884 S: * VANISHED 1:5,7:8,10:15 885 S: A142 OK [READ-WRITE] SELECT completed 887 6. Formal Syntax 889 The following syntax specification uses the Augmented Backus-Naur 890 Form (ABNF) notation as specified in [ABNF]. 892 Non-terminals referenced but not defined below are as defined by 893 [RFC3501], [CONDSTORE] or [IMAPABNF]. 895 Except as noted otherwise, all alphabetic characters are case- 896 insensitive. The use of upper or lower case characters to define 897 token strings is for editorial clarity only. Implementations MUST 898 accept these strings in a case-insensitive fashion. 900 capability =/ "X-DRAFT-W02-QRESYNC" 901 ;; [[Note to RFC Editor: fix before 902 ;; publication]] 904 select-param = "QRESYNC" SP "(" uidvalidity SP 905 mod-sequence-value [SP known-uids] 906 [SP seq-match-data] ")" 907 ;; conforms to the generic select-param 908 ;; syntax defined in [IMAPABNF] 910 seq-match-data = "(" known-sequence-set SP known-uid-set ")" 912 uidvalidity = nz-number 913 known-uids = sequence-set 914 ;; sequence of UIDs, "*" is not allowed 916 known-sequence-set = sequence-seq 917 ;; set of message numbers corresponding to 918 ;; the UIDs in known-uid-set, in ascending order. 919 ;; * is not allowed. 921 known-uid-set = sequence-seq 922 ;; set of UIDs corresponding to the messages in 923 ;; known-sequence-set, in ascending order. 924 ;; * is not allowed. 926 message-data =/ expunged-resp 928 expunged-resp = "VANISHED" [expunge-correlator] SP known-uids 930 expunge-correlator = SP "(" single-exp-correlator 931 *(SP single-exp-correlator) ")" 932 ;; Unless explicitly specified otherwise, 933 ;; all correlator types must be specified 934 ;; only once. 936 single-exp-correlator = "TAG" SP tag-string 937 ;; Correlator type followed by parameters 939 rexpunges-fetch-mod = "VANISHED" 940 ;; VANISHED UID FETCH modifier conforms 941 ;; to the fetch-modifier syntax 942 ;; defined in [IMAPABNF]. It is only 943 ;; allowed in the UID FETCH command. 945 7. Security Considerations 947 As always, it is important to thoroughly test clients and servers 948 implementing this extension, as it changes how the server reports 949 expunged messages to the client. 951 Security considerations relevant to [CONDSTORE] are relevant to this 952 extension. 954 This document doesn't raise any new security concerns not already 955 raised by [CONDSTORE] or [RFC3501]. 957 8. IANA Considerations 959 IMAP4 capabilities are registered by publishing a standards track or 960 IESG approved experimental RFC. The registry is currently located 961 at: 963 http://www.iana.org/assignments/imap4-capabilities 965 This document defines the X-DRAFT-W02-QRESYNC [[anchor21: The final 966 capability name will be chosen during AUTH48]] IMAP capability. IANA 967 is requested to add this capability to the registry. 969 9. Acknowledgments 971 Thanks to Steve Hole, Cyrus Daboo and Michael Wener for encouraging 972 creation of this document. 974 Valuable comments, both in agreement and in dissent, were received 975 from Timo Sirainen, Michael Wener, Randall Gellens, Arnt Gulbrandsen, 976 Peter Coates, Mark Crispin and Elwyn Davies. 978 This document takes substantial text from [RFC3501] by Mark Crispin. 980 10. References 982 10.1. Normative References 984 [ABNF] Crocker, D., Ed. and P. Overell, Ed., "Augmented BNF for 985 Syntax Specifications: ABNF", RFC 4234, October 2005. 987 [CONDSTORE] 988 Melnikov, A. and S. Hole, "IMAP Extension for Conditional 989 STORE Operation or Quick Flag Changes Resynchronization", 990 RFC 4551, June 2006. 992 [IMAPABNF] 993 Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 994 ABNF", RFC 4466, April 2006. 996 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate 997 Requirement Levels", BCP 14, RFC 2119, March 1997. 999 [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 1000 4rev1", RFC 3501, March 2003. 1002 [UIDPLUS] Crispin, M., "Internet Message Access Protocol (IMAP) - 1003 UIDPLUS extension", RFC 4315, December 2005. 1005 10.2. Informative References 1007 [IMAP-DISC] 1008 Melnikov, A., "Synchronization Operations For Disconnected 1009 Imap4 Clients", RFC 4549, June 2006. 1011 Authors' Addresses 1013 Alexey Melnikov 1014 Isode Ltd 1015 5 Castle Business Village 1016 36 Station Road 1017 Hampton, Middlesex TW12 2BX 1018 UK 1020 Email: Alexey.Melnikov@isode.com 1022 Dave Cridland 1023 Isode Ltd 1024 5 Castle Business Village 1025 36 Station Road 1026 Hampton, Middlesex TW12 2BX 1027 UK 1029 Email: dave.cridland@isode.com 1031 Corby Wilson 1032 Nokia 1033 5 Wayside Rd. 1034 Burlington, MA 01803 1035 USA 1037 Email: corby@computer.org 1039 Full Copyright Statement 1041 Copyright (C) The Internet Society (2006). 1043 This document is subject to the rights, licenses and restrictions 1044 contained in BCP 78, and except as set forth therein, the authors 1045 retain all their rights. 1047 This document and the information contained herein are provided on an 1048 "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS 1049 OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET 1050 ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, 1051 INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE 1052 INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 1053 WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 1055 Intellectual Property 1057 The IETF takes no position regarding the validity or scope of any 1058 Intellectual Property Rights or other rights that might be claimed to 1059 pertain to the implementation or use of the technology described in 1060 this document or the extent to which any license under such rights 1061 might or might not be available; nor does it represent that it has 1062 made any independent effort to identify any such rights. Information 1063 on the procedures with respect to rights in RFC documents can be 1064 found in BCP 78 and BCP 79. 1066 Copies of IPR disclosures made to the IETF Secretariat and any 1067 assurances of licenses to be made available, or the result of an 1068 attempt made to obtain a general license or permission for the use of 1069 such proprietary rights by implementers or users of this 1070 specification can be obtained from the IETF on-line IPR repository at 1071 http://www.ietf.org/ipr. 1073 The IETF invites any interested party to bring to its attention any 1074 copyrights, patents or patent applications, or other proprietary 1075 rights that may cover technology that may be required to implement 1076 this standard. Please address the information to the IETF at 1077 ietf-ipr@ietf.org. 1079 Acknowledgment 1081 Funding for the RFC Editor function is provided by the IETF 1082 Administrative Support Activity (IASA).