ietf
[Top] [All Lists]

Re: Best practice for data encoding?

2006-06-08 00:50:42
On Wed Jun  7 23:19:29 2006, Iljitsch van Beijnum wrote:
On 7-jun-2006, at 22:38, Dave Cridland wrote:

I think it's worth noting that nobody is preventing you from using XML over a compressed channel, which tends to increase XML's efficiency rather sharply.

[...]

Wire efficiency, for the most part, needs to take place by avoiding the transmission of information entirely, rather than trying to second-guess the compression.

Obviously adding compression doesn't help processing efficiency.


It does if you're doing encryption, unless the protocol itself is almost entirely uncompressible. Compression is a lot cheaper than encryption, and the more you compress, the less you have to encrypt.


I've long harbored the suspicion that a large percentage of the cycles in today's fast CPUs are burned up parsing various types of text. Does anyone know of any processing efficiency comparisons between binary and text based protocols?


But it's a clear trade-off between efficient representation and complexity for the developer - we've had this debate once already in this thread, I think. The more complex the representation, the harder it is to code correctly and debug.

I'm not saying there's no place for binary protocols, but I am saying that to go for a binary representation, you have to be working in a highly resource constrained environment where both bandwidth and CPU usage is highly limited.


As an example, IMAP and ACAP streams compress by around 70% on my client - and that's trying to be bandwidth efficient in its protocol usage. I've seen figures of 85% talked about quite seriously, too.

And you think that's a good thing?

No, it demonstrates that even protocols which have a reputation for chatter actually turn out to be efficient on the wire post-compression.

Now I understand why it takes me up to 10 minutes to download a handful of 1 - 2 kbyte emails with IMAP (+SSL) over a 9600 bps GSM data connection.

I can write an efficient IMAP client, and give it away for free, but I can't force you to use it. :-)

I can read a new message inside a minute over 9600bps with 1 second latency, without even using a local disk cache - including fetching configuration over ACAP to begin with. My INBOX contains 24,033 messages currently - obviously I'm not downloading them all, and this is emulated bandwidth and latency, thus not entirely accurate.

IMAP can be extremely efficient; if it's taking you 10 minutes to read a handful of emails, you're using the wrong client, the wrong server, or quite possibly both. With Lemonade features coming into mainstream implementations, you really ought to be looking at your systems if you're using GSM data. Compression is not really one of these, oddly enough, but TLS deflate ought to be available to most clients and servers by now anyway.

Dave.
--
Dave Cridland - mailto:dave(_at_)cridland(_dot_)net - 
xmpp:dwd(_at_)jabber(_dot_)org
 - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
 - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

_______________________________________________
Ietf mailing list
Ietf(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/ietf