Re: [EAI] Body parts

[John C Klensin <klensin at jck.com>]

Shawn,

I think this is out of scope for the WG so, until Harald or
XiaoDong tell me otherwise, I'm going to try to make this my
last note on the subject (or take it off list).   But...

--On Saturday, 12 July, 2008 21:41 -0700 Shawn Steele
<Shawn.Steele at microsoft.com> wrote:

>> Really?  Your data collection sources obviously differ from
>> mine, but my impression is that we've seen very little
>> transmission loss or confusion in the last 10 or 15 years, at
>> least among MIME-conformant implementations that follow the
>> rules.
> 
> Well, of course if they all play well it should work :)
> 
> The Unicode list gets mail fairly regularly that someone
> complains they couldn't read :)

>From when I was trying to follow that list, as well as general
experience, I would suggest that the origin of most of those
problems are not ones in which the content-type and associated
charset parameter match the actual content of the message.  They
are because a sending MUA (or web interface configuration) has a
default charset label that it applies, one that isn't UTF-8 or
because the type of copying and pasting that I described in my
note to Arnt has occurred (most often just by quoting all or
part of a reply).
 
> Remember that I'm suggesting this for apps advertising
> UTF8STMP, so there *should* be no loss of data by complying
> applications, since presumably they'd all be designed
> correctly.  For the misbehaving ones, as you pointed out
> there's already a problem.

Actually, the fact that a particular app can handle UTF-8 for
headers is not really a sufficient condition given the way may
mail systems are organized.  If a system receives mail in one
charset and format, and the user wants to paste part of its
contents into another message, ability to convert that character
set to UTF-8 and do so at the right point is required, not
merely the ability to render both charsets.  And the "at the
right point" part is significant too, as our documents on the
use of POP and IMAP with UTF8 headers and envelopes demonstrate.

> I hear 99% of the code page related issues within Microsoft,
> and I've encountered difficulties with code pages from
> Hotmail, Live Mail, Exchange & Outlook.  Many of those are
> probably when receiving mail without proper MIME content-type
> information. 

Of course, historically, some of those systems have been among
the worst offenders about not properly identifying the charsets,
code pages, and file types they are sending out.

> Other causes seem to be varying code page
> implementations between the sender & recipient &/or different
> interpretations of the standards or code page names.

Possibly, but I've seen symptoms of that far less than the above.

> Obviously recommending UTF-8 can't stop mail clients from
> doing the wrong thing, but if we could recommend a course that
> moves towards a single encoding, then the entire
> encoding-related class of issues should eventually start doing
> better.  As someone mentioned UTF-8 is being adopted by more
> mail clients, but it can't hurt to inspire people to support
> UTF-8.

I hate to say "cleaning up your own house would make more
difference than anything the IETF could do", but, if it were
made very difficult to transmit anything textual from, e.g.,
Outlook (both current and, via service packs, back to at least
to the 2003 version) in other than UTF-8, with content-type
fields (including the charset parameter) and file name fields
all consistent with the UTF-8 being transmitted, I'd guess a
large fraction of the problem would disappear in a fairly short
time.  

On the other hand, if we "merely" change the Standard, I note
that one major MUA with a very long history still does not
handle MIME properly (it uses an "attachment" model rather than
a "body part" one, leading to both subtle and gross problems)
and that the contemporary version of Office doesn't precisely
follow the standard or encourage users strongly enough to
configure systems to do so (e.g., too many things sent as
application/octet-stream who identity is perfectly well know to
the Operating System).

> FWIW: Mark Davis was happy to show that UTF-8 is now the most
> common web encoding.
> http://www.w3.org/QA/2008/05/utf8-web-growth.html Sure, web
> isn't mail, but it does reflect a general trend.

See SM's note.

   john


_______________________________________________
IMA mailing list
IMA at ietf.org
https://www.ietf.org/mailman/listinfo/ima