Re: Best practice for data encoding?

[Dave Cridland <dave at cridland.net>]

On Wed Jun  7 23:19:29 2006, Iljitsch van Beijnum wrote:
> On 7-jun-2006, at 22:38, Dave Cridland wrote:
>
> > I think it's worth noting that nobody is preventing you from using 
> >  XML over a compressed channel, which tends to increase XML's  
> > efficiency rather sharply.
>
> [...]
>
> > Wire efficiency, for the most part, needs to take place by 
> > avoiding  the transmission of information entirely, rather than 
> > trying to  second-guess the compression.
>
> Obviously adding compression doesn't help processing efficiency.
It does if you're doing encryption, unless the protocol itself is 
almost entirely uncompressible. Compression is a lot cheaper than 
encryption, and the more you compress, the less you have to encrypt.


> I've long harbored the suspicion that a large percentage of the  
> cycles in today's fast CPUs are burned up parsing various types of  
> text. Does anyone know of any processing efficiency comparisons  
> between binary and text based protocols?
But it's a clear trade-off between efficient representation and 
complexity for the developer - we've had this debate once already in 
this thread, I think. The more complex the representation, the harder 
it is to code correctly and debug.

I'm not saying there's no place for binary protocols, but I am saying 
that to go for a binary representation, you have to be working in a 
highly resource constrained environment where both bandwidth and CPU 
usage is highly limited.


> > As an example, IMAP and ACAP streams compress by around 70% on my  
> > client - and that's trying to be bandwidth efficient in its  
> > protocol usage. I've seen figures of 85% talked about quite  
> > seriously, too.
>
> And you think that's a good thing?

No, it demonstrates that even protocols which have a reputation for 
chatter actually turn out to be efficient on the wire 
post-compression.

>  Now I understand why it takes me  up to 10 minutes to download a 
> handful of 1 - 2 kbyte emails with  IMAP (+SSL) over a 9600 bps GSM 
> data connection.

I can write an efficient IMAP client, and give it away for free, but 
I can't force you to use it. :-)

I can read a new message inside a minute over 9600bps with 1 second 
latency, without even using a local disk cache - including fetching 
configuration over ACAP to begin with. My INBOX contains 24,033 
messages currently - obviously I'm not downloading them all, and this 
is emulated bandwidth and latency, thus not entirely accurate.

IMAP can be extremely efficient; if it's taking you 10 minutes to 
read a handful of emails, you're using the wrong client, the wrong 
server, or quite possibly both. With Lemonade features coming into 
mainstream implementations, you really ought to be looking at your 
systems if you're using GSM data. Compression is not really one of 
these, oddly enough, but TLS deflate ought to be available to most 
clients and servers by now anyway.

Dave.
--
Dave Cridland - mailto:dave at cridland.net - xmpp:dwd at jabber.org
 - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
 - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

_______________________________________________
Ietf mailing list
Ietf at ietf.org
https://www1.ietf.org/mailman/listinfo/ietf