idnits 2.17.1 draft-ietf-imapext-thread-02.txt: Checking boilerplate required by RFC 5378 and the IETF Trust (see https://trustee.ietf.org/license-info): ---------------------------------------------------------------------------- ** Looks like you're using RFC 2026 boilerplate. This must be updated to follow RFC 3978/3979, as updated by RFC 4748. Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: ---------------------------------------------------------------------------- ** Missing expiration date. The document expiration date should appear on the first and last page. ** The document seems to lack a 1id_guidelines paragraph about the list of Shadow Directories. == No 'Intended status' indicated for this document; assuming Proposed Standard == It seems as if not all pages are separated by form feeds - found 0 form feeds but 12 pages Checking nits according to https://www.ietf.org/id-info/checklist : ---------------------------------------------------------------------------- ** The document seems to lack an Introduction section. ** The document seems to lack an IANA Considerations section. (See Section 2.2 of https://www.ietf.org/id-info/checklist for how to handle the case when there are no actions for IANA.) ** The abstract seems to contain references ([ABNF], [NEWS]), which it shouldn't. Please replace those with straight textual mentions of the documents in question. ** The document seems to lack a both a reference to RFC 2119 and the recommended RFC 2119 boilerplate, even if it appears to use RFC 2119 keywords. RFC 2119 keyword, line 82: '...onnected clients MUST use exactly this...' RFC 2119 keyword, line 199: '...rent/child links MUST be kept consiste...' RFC 2119 keyword, line 418: '...plementations of THREAD MUST implement...' Miscellaneous warnings: ---------------------------------------------------------------------------- -- The document seems to lack a disclaimer for pre-RFC5378 work, but may have content which was first submitted before 10 November 2008. If you have contacted all the original authors and they are all willing to grant the BCP78 rights to the IETF Trust, then this is fine, and you can ignore this comment. If not, you may need to add the pre-RFC5378 disclaimer. (See the Legal Provisions document at https://trustee.ietf.org/license-info for more information.) -- The document date (August 2000) is 8648 days in the past. Is this intentional? Checking references for intended status: Proposed Standard ---------------------------------------------------------------------------- (See RFCs 3967 and 4897 for information about using normative references to lower-maturity documents in RFCs) -- Missing reference section? 'ABNF' on line 450 looks like a reference -- Missing reference section? 'NEWS' on line 453 looks like a reference Summary: 7 errors (**), 0 flaws (~~), 2 warnings (==), 4 comments (--). Run idnits with the --verbose option for more detailed information about the items above. -------------------------------------------------------------------------------- 2 IMAP Extensions Working Group M. Crispin 3 Internet Draft: IMAP THREAD K. Murchison 4 Document: internet-drafts/draft-ietf-imapext-thread-02.txt August 2000 6 INTERNET MESSAGE ACCESS PROTOCOL - THREAD EXTENSION 8 Status of this Memo 10 This document is an Internet-Draft and is in full conformance with 11 all provisions of Section 10 of RFC 2026. 13 Internet-Drafts are working documents of the Internet Engineering 14 Task Force (IETF), its areas, and its working groups. Note that 15 other groups may also distribute working documents as Internet- 16 Drafts. 18 Internet-Drafts are draft documents valid for a maximum of six months 19 and may be updated, replaced, or obsoleted by other documents at any 20 time. It is inappropriate to use Internet-Drafts as reference 21 material or to cite them other than as "work in progress." 23 The list of current Internet-Drafts can be accessed at 24 http://www.ietf.org/ietf/1id-abstracts.txt 26 To view the list Internet-Draft Shadow Directories, see 27 http://www.ietf.org/shadow.html. 29 A revised version of this draft document will be submitted to the RFC 30 editor as a Proposed Standard for the Internet Community. 32 Discussion and suggestions for improvement are requested, and should 33 be sent to ietf-imapext@IMC.ORG. This document will expire before 1 34 March 2001. Distribution of this memo is unlimited. 36 Abstract 38 This document describes the server-based threading extension to the 39 IMAP4rev1 protocol. This extension provides substantial performance 40 improvements for IMAP clients which offer threaded views. 42 A server which supports this extension indicates this with more or 43 more capability names consisting of "THREAD=" followed by a supported 44 threading algorithm name as described in this document. This 45 provides for future upwards-compatible extensions. 47 Extracted Subject Text 49 Threading uses a version of the subject which has specific subject 50 artifacts of deployed Internet mail software removed. Due to the 51 complexity of these artifacts, the formal syntax for the subject 52 extraction rules is ambiguous. The following procedure is followed 53 to determining the actual "base subject" which is used to thread: 55 (1) Convert any RFC 2047 encoded-words in the subject to 56 UTF-8. Convert all tabs and continuations to space. 57 Convert all multiple spaces to a single space. 59 (2) Remove all trailing text of the subject that matches 60 the subj-trailer ABNF, repeat until no more matches are 61 possible. 63 (3) Remove all prefix text of the subject that matches the 64 subj-leader ABNF. 66 (4) If there is prefix text of the subject that matches the 67 subj-blob ABNF, and removing that prefix leaves a non-empty 68 subj-base, then remove the prefix text. 70 (5) Repeat (3) and (4) until no matches remain. 72 Note: it is possible to defer step (2) until step (6), but this 73 requires checking for subj-trailer in step (4). 75 (6) If the resulting text begins with the subj-fwd-hdr ABNF 76 and ends with the subj-fwd-trl ABNF, remove the 77 subj-fwd-hdr and subj-fwd-trl and repeat from step (2). 79 (7) The resulting text is the "base subject" used in 80 threading. 82 All servers and disconnected clients MUST use exactly this algorithm 83 when threading. Otherwise there is potential for a user to get 84 inconsistant results based on whether they are running in connected 85 or disconnected IMAP mode. 87 Additional Commands 89 This command is an extension to the IMAP4rev1 base protocol. 91 The section header is intended to correspond with where it would be 92 located in the main document if it was part of the base 93 specification. 95 6.3.THREAD. THREAD Command 97 Arguments: threading algorithm 98 charset specification 99 searching criteria (one or more) 101 Data: untagged responses: THREAD 103 Result: OK - thread completed 104 NO - thread error: can't thread that charset or 105 criteria 106 BAD - command unknown or arguments invalid 108 The THREAD command is a variant of SEARCH with threading semantics 109 for the results. Thread has two arguments before the searching 110 criteria argument; a threading algorithm, and the searching 111 charset. Note that unlike SEARCH, the searching charset argument 112 is mandatory. 114 There is also a UID THREAD command which corresponds to THREAD the 115 way that UID SEARCH corresponds to SEARCH. 117 The THREAD command first searches the mailbox for messages that 118 match the given searching criteria using the charset argument for 119 the interpretation of strings in the searching criteria. It then 120 returns the matching messages in an untagged THREAD response, 121 threaded according to the specified threading algorithm. 123 The defined threading algorithms are as follows: 125 ORDEREDSUBJECT 126 The ORDEREDSUBJECT threading algorithm is also referred to as 127 "poor man's threading." The searched messages are sorted by 128 subject and then by sent date, equivalent to a "SORT (SUBJECT 129 DATE)". The messages are then split into separate threads, 130 with each thread containing messages with the same extracted 131 subject text. Finally, the threads are sorted by the sent date 132 of the first message in the thread. 134 Note that each message in a thread is a child (as opposed to a 135 sibling) of the previous message. 137 REFERENCES 138 The REFERENCES threading algorithm is based on the algorithm 139 written by Jamie Zawinski which was used in "Netscape Mail and 140 News" versions 2.0 through 3.0. For details, see 141 http://www.jwz.org/docs/threading.html. 143 This algorithm threads the searched messages by grouping them 144 together in parent/child relationships based on which messages 145 are replies to others. The parent/child relationships are 146 built using two methods: reconstructing a message's ancestry 147 using the references contained within it; and checking the 148 subject of a message to see if it is a reply to (or forward of) 149 another. 151 The references used for reconstructing a message's ancestry are 152 found using the following rules: 154 If a message contains a [NEWS]-style References header, then 155 use the Message-IDs in this header as the references. 157 If a message does not contain a References header, or the 158 header does not contain any Message-IDs, then use the first 159 (if any) Message-ID found in the In-Reply-To header as the 160 only reference (parent) for this message. 162 If a message does not contain an In-Reply-To header, or the 163 header does not contain a Message-ID, then the message does 164 not have any references (NIL). 166 The REFERENCES algorithm is significantly more complex than 167 ORDEREDSUBJECT and consists of five main steps. These steps 168 are outlined in detail below. 170 (1) For each searched message: 172 (A) Using the Message-IDs in the message's references, link 173 the corresponding messages together as parent/child. Make 174 the first reference the parent of the second (and the second 175 a child of the first), the second the parent of the third 176 (and the third a child of the second), etc. The following 177 rules govern the creation of these links: 179 If no reference message can be found with a given 180 Message-ID, create a dummy message with this ID. Use 181 this dummy message for all subsequent references to this 182 ID. 184 If a reference message already has a parent, don't change 185 the existing link. 187 Do not create a parent/child link if creating that link 188 would introduce a loop. For example, before making 189 message A the parent of B, make sure that A is not a 190 descendent of B. 192 (B) Create a parent/child link between the last reference 193 (or NIL if there are no references) and the current message. 194 If the current message has a parent already, break the 195 current parent/child link before creating the new one. Note 196 that if this message has no references, that it will now 197 have no parent. 199 NOTE: The parent/child links MUST be kept consistent with 200 one another at ALL times. 202 (2) Gather together all of the messages that have no parents 203 and make them all children (siblings of one another) of a dummy 204 parent (the "root"). These messages constitute first messages 205 of the threads created thus far. 207 (3) Prune dummy messages from the thread tree. Traverse each 208 thread under the root, and for each message: 210 If it is a dummy message with NO children, delete it. 212 If it is a dummy message with children, delete it, but 213 promote its children to the current level. In other words, 214 splice them in with the dummy's siblings. 216 Do not promote the children if doing so would make them 217 children of the root, unless there is only one child. 219 (4) Gather together messages under the root that have the same 220 extracted subject text. 222 (A) Create a table for associating extracted subjects with 223 messages. 225 (B) Populate the subject table with one message per 226 extracted subject. For each message under the root: 228 (i) Find the subject of this thread by extracting the 229 base subject from the current message, or its first child 230 if the current message is a dummy. 232 (ii) Lookup the message associated with this extracted 233 subject in the table. 235 (iii) If there is no message in the table with this 236 subject, add the current message and the extracted 237 subject to the subject table. 239 Otherwise, replace the message in the table with the 240 current message if either of the following criteria are 241 true: 243 The current message is a dummy and the one in the 244 table is not, OR 246 The message in the table is a reply or forward (its 247 original subject contains a subj-refwd part and/or a 248 "(fwd)" subj-trailer) and the current message is not. 250 (C) Merge threads with the same subject. For each message 251 under the root: 253 (i) Find the subject of this thread as in step 4.B.i 254 above. 256 (ii) Lookup the message associated with this extracted 257 subject in the table. 259 (iii) If the message in the table is the current message, 260 skip it. 262 Otherwise, merge the current message with the one in the 263 table using the following rules: 265 If both messages are dummies, append the current 266 message's children to the children of the message in 267 the table (the children of both messages become 268 siblings), and then delete the current message. 270 If the message in the table is a dummy and the current 271 message is not, make the current message a child of 272 the message in the table (a sibling of it's children). 274 If the current message is a reply or forward and the 275 message in the table is not, make the current message 276 a child of the message in the table (a sibling of it's 277 children). 279 Otherwise, create a new dummy container and make both 280 messages children of the dummy, and replace the 281 message in the table with the dummy message. 283 (5) Traverse the messages under the root and sort each set of 284 siblings by date. In the case of an exact match on date, use 285 the order in which they appear in the mailbox (that is, by 286 sequence number) to determine the order. In the case of a 287 dummy message (which can occur with top-level siblings), use 288 the first child for sorting. 290 Example: C: A283 THREAD ORDEREDSUBJECT UTF-8 SINCE 5-MAR-2000 291 S: * THREAD (166)(167)(168)(169)(172)(170)(171) 292 (173)(174 175 176 178 181 180)(179)(177 183 293 182 188 184 185 186 187 189)(190)(191)(192) 294 (193)(194 195)(196 197 198)(199)(200 202)(201) 295 (203)(204)(205)(206 207)(208) 296 S: A283 OK THREAD completed 297 C: A284 THREAD ORDEREDSUBJECT US-ASCII TEXT "gewp" 298 S: * THREAD 299 S: A284 OK THREAD completed 300 C: A285 THREAD REFERENCES UTF-8 SINCE 5-MAR-2000 301 S: * THREAD (166)(167)(168)(169)(172)((170)(179)) 302 (171)(173)((174)(175)(176)(178)(181)(180)) 303 ((177)(183)(182)(188 (184)(189))(185 186)(187)) 304 (190)(191)(192)(193)((194)(195 196))(197 198) 305 (199)(200 202)(201)(203)(204)(205 206 207)(208) 306 S: A285 OK THREAD completed 308 Note: The line breaks in the first and third client 309 responses are for editorial clarity and do not appear in 310 real THREAD responses. 312 Additional Responses 314 This response is an extension to the IMAP4rev1 base protocol. 316 The section heading of this response is intended to correspond with 317 where it would be located in the main document. 319 7.2.THREAD. THREAD Response 321 Data: zero or more threads 323 The THREAD response occurs as a result of a THREAD or UID THREAD 324 command. It contains zero or more threads. A thread consists of 325 a parenthesized list of thread members. 327 Thread members consist of zero or more message numbers, delimited 328 by spaces, indicating successive parent and child. This continues 329 until the thread splits into multiple sub-threads, at which point 330 the thread nests into multiple sub-threads with the first member 331 of each subthread being siblings at this level. There is no limit 332 to the nesting of threads. 334 The messages numbers refer to those messages that match the search 335 criteria. For THREAD, these are message sequence numbers; for UID 336 THREAD, these are unique identifiers. 338 Example: S: * THREAD (2)(3 6 (4 23)(44 7 96)) 340 The first thread consists only of message 2. The second thread 341 consists of the messages 3 (parent) and 6 (child), after which it 342 splits into two subthreads; the first of which contains messages 4 343 (child of 6, sibling of 44) and 23 (child of 4), and the second of 344 which contains messages 44 (child of 6, sibling of 4), 7 (child of 345 44), and 96 (child of 7). Since some later messages are parents 346 of earlier messages, the messages were probably moved from some 347 other mailbox at different times. 349 -- 2 351 -- 3 352 \-- 6 353 |-- 4 354 | \-- 23 355 | 356 \-- 44 357 \-- 7 358 \-- 96 360 Example: S: * THREAD ((3)(5)) 362 In this example, 3 and 5 are siblings of a parent which does not 363 exist in the mailbox; however they are members of the same thread. 365 Formal Syntax of THREAD commands and Responses 367 thread-data = "THREAD" [SP 1*thread-list] 369 thread-list = "(" thread-members / thread-nested ")" 371 thread-members = nz-number *(SP nz-number) [SP thread-nested] 373 thread-nested = 2*thread-list 375 thread = ["UID" SP] "THREAD" SP thread-algorthm 376 SP search-charset 1*(SP search-key) 378 thread-algorithm = "ORDEREDSUBJECT" / "REFERENCES" / atom 380 The following syntax describes subject extraction rules (2)-(6): 382 subject = *subj-leader [subj-middle] *subj-trailer 384 subj-refwd = ("re" / ("fw" ["d"])) *WSP [subj-blob] ":" 386 subj-blob = "[" *BLOBCHAR "]" *WSP 388 subj-fwd = subj-fwd-hdr subject subj-fwd-trl 390 subj-fwd-hdr = "[fwd:" 392 subj-fwd-trl = "]" 394 subj-leader = (*subj-blob subj-refwd) / WSP 396 subj-middle = *subj-blob (subj-base / subj-fwd) 397 ; last subj-blob is subj-base if subj-base would 398 ; otherwise be empty 400 subj-trailer = "(fwd)" / WSP 402 subj-base = NONWSP *([*WSP] NONWSP) 403 ; can be a subj-blob 405 BLOBCHAR = %x01-5a / %x5c / %x5e-7f 406 ; any CHAR except '[' and ']' 408 NONWSP = %x01-08 / %x0a-1f / %x21-7f 409 ; any CHAR other than WSP 411 Security Considerations 413 Security issues are not discussed in this memo. 415 Internationalization Considerations 417 By default, strings are threaded according to the "minimum sorting 418 collation algorithm". All implementations of THREAD MUST implement 419 the minimum sorting collation algorithm. 421 In the minimum sorting collation algorithm, the 26 Latin alphabetics 422 are sorted in a case-insensitive fashion; that is, "A" and "a" are 423 treated as exact equals. All other characters are sorted according 424 to their octet values, as expressed in UTF-8. No attempt is made to 425 treat composed characters specially. 427 Other sorting collations, and the ability to change the sorting 428 collation, will be defined in a separate document dealing with IMAP 429 internationalization. 431 It is anticipated that there will be a generic Unicode sorting 432 collation, which will provide generic case-insensitivity for 433 alphabetic scripts, specification of composed character handling, and 434 language-specific sorting collations. A server which implements 435 non-default sorting collations will modify its sorting behavior 436 according to the selected sorting collation. 438 Non-English translations of "Re" or "Fw"/"Fwd" are not specified for 439 removal in the extracted subject text process. By specifying that 440 only the English forms of the prefixes are used, it becomes a simple 441 display time task to localize the prefix language for the user. If, 442 on the other hand, prefixes in multiple languages are permitted, the 443 result is a geometrically complex, and ultimately unimplementable, 444 task. In order to improve the ability to support non-English display 445 in Internet mail clients, only the English form of these prefixes 446 should be transmitted in Internet mail messages. 448 A. References 450 [ABNF] Crocker, D., and Overell, P. "Augmented BNF for Syntax 451 Specifications: ABNF", RFC 2234, November 1997. 453 [NEWS] Horton, M., and Adams, R., "Standard for interchange of USENET 454 messages", RFC-1036, AT&T Bell Laboratories and Center for Seismic 455 Studies, December, 1987. 456 Author's Address 458 Mark R. Crispin 459 Networks and Distributed Computing 460 University of Washington 461 4545 15th Avenue NE 462 Seattle, WA 98105-4527 464 Phone: (206) 543-5762 466 EMail: MRC@CAC.Washington.EDU 468 Kenneth Murchison 469 Oceana Matrix Ltd. 470 21 Princeton Place 471 Orchard Park, NY 14127 473 Phone: (716) 662-8973 x26 475 EMail: ken@oceana.com