[MORG] Why IMAP extensions are (not) used

[Arnt Gulbrandsen <arnt at oryx.com>]

Arnt Gulbrandsen writes:
> Curtis King writes:
> > I'm sure there are some more clients which use a few more extensions, 
> > but that still means ~16 extensions we (Isode) have implemented are 
> > not being used. Not a very good ratio. Yet, here we are planning on 
> > adding more.
>
> A bit of discussion of the reasons seems like a good idea. I have 
> opinions but it's late and I'm too tired now. I'll try to post 
> something during the coming week.

Here goes. I'm sure someone will tell me I'm all wrong. Go on, but 
please do say something about what's right too.

I believe that IMAP extensions aren't used as much as they should be for 
a combination of five reasons. I believe that all five are factors in 
every case, but the weight varies widely.

Further, I believe that all five can be mitigated, in some cases quite 
easily. Mitigating is not the same as eliminating, and of course 
mitigating the known problems is enough to ensure takeup of a 
particular new extension.

1. RFCs written with the wrong audience in mind.

Many of the RFCs don't make much effort to serve client authors or 
answer their questions, such as «how should I use this?», «how should I 
fall back when it isn't supported?». Some RFCs have few examples, some 
have many, but «many» is not the same as «useful for a particular 
audience».

(Without making a detailed survey, I'd guess that the ones with many are 
fairly good for server implementers, less so for clients. Server 
implementers need more text and examples about corner cases, clients 
more that's related to their use cases.)

2. RFCs can be hard to find.

We don't have a list mapping from problem/benefit to the relevant 
extension(s), and googling doesn't solve that problem well.

3. What if/ifnot.

Clients generally need to implement stuff both for servers that support 
a given extensions, and for servers that don't. We haven't given much 
thought to minimising the duplicated code paths that result, or do it 
cleanly.

(I don't mean textually this time, I mean functionally.)

4. Uncertainty about compliant servers.

It's needlessly difficult for a client to find out which servers support 
something, which makes it harder to test with/without a particular 
extension than it should be, and also harder to know whether 
implementing an extension will have a benefit in the wild.

Another aspect of this is interoperation.

5. Combinatorial behaviour.

Several non-IETF people I've spoken to complain about the number of 
combinations.

The number of combinations is effectively higher than the number of 
extensions, since some extensions interact in tricky ways. This strikes 
me as wrong, and most of the interactions are IMO just neat-to-have.

It could be lower in practice, since a client can always do
  if ( server supports a, b and c)
     use all three
  else
     use none

We could try harder to avoid the interactions, and perhaps occasionally 
call attention to the a && b && c trick. (That trick also needs 
information about how many servers implement a and b but not c, etc.)

Arnt

(PS: I've tried to avoid digression in this little missive, as I think 
we would otherwise get bogged down the third sentence of RFC x section 
y.x, instead of why IMAP discussions aren't used as much as we'd like 
and how MORG can have a good fate.)