Re: [MORG] Why IMAP extensions are (not) used

[Mark Crispin <markrcrispin at panda.com>]

On Wed, 8 Apr 2009, Barry Leiba wrote:
> If you have (say) a DB2 back-end to your mail store, a search of 350
> mailboxes might be very efficient.

At some other costs... ;-)

> If you have (say) a set of Unix mailboxes as the back-end to your mail
> store, a search of 350 mailboxes might be pretty much the same,
> whether you do it in one command or 700, apart from the protocol
> overhead.

Note that to do IMAP SEARCH right, you have to be cognizant of the MIME 
structure, select only the textual parts, and search the textual data 
using the i;unicode-casemap algorithm.  "Just grep through the mailboxes" 
doesn't cut it any more.  grep doesn't know about a string that may be in 
a non-UTF8 charset within BASE64 encoded content, or about text vs. image 
MIME parts.

Either you do the necessary computations on the fly at demand, or do the 
calculations in advance and store the results separate from the message 
data, or do something in between.

No database can somehow make this choice go away.  The form may change, 
but not the function.

Nor is it as easy as saying "we have lots of disk space, just precompute 
and store everything".  The more you store, the more complexity that you 
have to deal with.  Also, delivery time is a really bad time to be doing 
anything time-consuming.

> Whatever the decision, Mark's point, which is correct, is that you
> can't make assumptions.  A client that wants to let a user search 350
> mailboxes will do it, one way or another.  But it might or might not
> perform badly with or without the extension, and you can't know in
> advance which it'll be.

There's more than that.  A server may decide that the client that searches 
350 mailboxes is abuse, and will "implement" the multimailbox search to 
block the abuse.  Only it won't really search.

Don't think for one minute that this isn't going on now.

For better or worse, most servers are designed with one goal, and only one 
goal, in mind: to implement synchronization with Outlook.  Everything 
else, and I do mean EVERYTHING else, is secondary.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.