[Top] [All Lists]

Re: [ietf-smtp] Proposal: SMTP Early Pipelining

2018-09-03 09:28:47
On 09/02/2018 04:38 PM, Ned Freed wrote:
FWIW, the IP address is the wrong key to use, since IP address can and do
change and/or get reassigned willy-nilly in some environments. The server host
name would be a better choice.

The name, on its own, has its own potential for problems in complex
environments.  Even a DNS-roundrobin to a mismatched set of servers.
On the plus side, a client might see fewer names than IPs.

Operators don't have to advertise the extension in such environments.
We could add guidance for cases where it won't work well.

But the problem here is that there's no means of invalidating the cache in the
event of a server capability change. This can lead to clients failing to use
extensions that are available, including security-related extensions, or
attempting to use extensions that are available. There are also extensions 
SIZE who values cannot be usefully cached.

This was the issue prompting the invalidate-cache-on-SMTP-error

I don't think that capability upgrades at a server are a big issue.
So long as the client does a time-based invalidate it will eventually
pick up the change, and in the meantime it is no worse off than an
unaware (of the new extension) client.

What this implies is that in order to be sure that things are OK, when the
synchronization point is reached prior to finalizing the transaction (and such
a point has to be included) the client must check and make sure the EHLO
response it got and the response it had cached are, modulo things like the
argument to the SIZE extension, the same in regards to the extensions the
client supports. If they aren't the client must then check and make sure none
of the changes mattered, and if they did, RSET and restart the transaction.
(The client also has the choice of playing it safe and restarting
unconditionally when the responses don't match.)

[Similar suggestion from John Klensin]

The downside is that we have to do the EHLO response parsing, previously
avoidable, but that is not a big point.  Overall I'm happy to add
this in as a MUST.

We might ask how often downgrades are expected.  I'd think "hardly
ever", outside the pathological environments considered above.

I'm afraid this part is a complete nonstarter. Spam engines do this routinely
and thus it's regularly used as a means of spam detection.

I fully agree that you don't want to advertise this globally.
Probably you already don't advertise PIPELINING globally for the
same reasons.  You may still make it available selectively.

 spam filtering SMTP proxies that
implement this sort of thing are common as dirt, and an MTA has no way 
of knowing if any of them are present.

I'm unsure why an MTA operator would not be aware of any proxies he
is operating.

And while we're considering security layers, it should also be noted that the
recent increase in use of STARTTLS has necessarily led to an overall increase
in round trips, yet AFAIK this increase had had no significant impact on SMTP
operationally. This argues strongly that the need for additional optimization
in this regard isn't exactly urgent.

I agree, it is a weak argument.  It is only efficiency and latency.

[I wrote:]
You can take 1.5 roundtrips out of a cleartext SMTP connection, 2.5
out of a STARTTLS one.

It combines nicely with TCP Fast-Open, as it converts the protocol
to client-starts.

Obviously, more roundtrips are saved when TFO is also used.

[John Klensin wrote:]
I remain to be convinced of the utility here. The PIPELINING extension was
written at a time when network latency was a much bigger problem than it is
now, the average number of recipients per message was higher, messages were on
average smaller, and the ability to use multiple connections in parallel was
much more limited. It was a clear win at the time, and in hindsight perhaps 
a bit too cautious in its approach.

Agreed on the number of recipients.  On the other hand, I think we see
more single-message connections (less batching).

On message size: yes it has increased - but for that we're into
bandwidth issues, not roundtrip time limitations; the data portion
of the SMTP connection is internally pipelined by TCP.


ietf-smtp mailing list