[Top] [All Lists]

Re: [ietf-smtp] Compressing SMTP streams

2016-01-29 17:30:33
On Fri, Jan 29, 2016 at 3:21 PM, Brandon Long <blong(_at_)google(_dot_)com> 

On Fri, Jan 29, 2016 at 1:42 PM, John Levine <johnl(_at_)taugh(_dot_)com> 

1000 recipients at ~30 bytes each with each domain likely the same...

Can you make any estimates of how common large recipient lists are?  My
impression is that since most bulk mail is customized per recipient, big
lists in stuff that isn't spam are rare.

Pretty rare, I think our average is in the 1.3 per message range.

Looking so far at today, apparently our 99th percentile is 1 and 99.9th is
7, so yeah, pretty rare.

If you're only compressing the data, what's the interaction with RFC 1870?
Do you use the compressed size or the original,

It has to be the original size.  If you're using a stream compressor
like deflate you can't tell what the compressed size will be unless
you make a prepass over the data.  And anyway, my impression is that
the motivation for using 1870 is still the internal storage, not the
transmission time.

Makes sense.

And I'd think it's an alternative to BDAT, same thing, but compressed?

Same problem, client generally doesn't know the compressed size in

Hmm, so we'd be stuck with trying to find the EOD marker, or doing some
streaming decompression til we reach the right end size.

I think I prefer the full stream compression.

And the nntp compress extension sounds also identical, a stream one.  Now,
nntp can be used in both an smtp like state between servers and an imap
like state from client to server, but total stream seems to be the common

I would be remiss if I didn't mention that one of my colleague's here at
Google thinks this is all silly, and we should instead figure out how to
not base64 encode the messages in the first place, but I think that's a
higher barrier than this.

ietf-smtp mailing list