[Top] [All Lists]

Re: Fwd: I-D ACTION:draft-hall-deferrals-00.txt

2007-02-04 04:49:13
On 2007-02-03 11:05:58 +0000, Tony Finch wrote:
On Fri, 2 Feb 2007, Eric A. Hall wrote:
On 1/30/2007 2:45 PM, Tony Finch wrote:
My main question about this draft is whether the extra complexity compared
to LMTP is really useful. What is your rationale for it?

I'd have to find my brain from 2003 to answer for sure, but I think the
motivation came down to the fact that LMTP is essentially designed as a
mailstore API and specifically was not designed as a transfer protocol (it
actually disavows all such purposes).

I believe that the rationale and limited purpose for which LMTP was
designed do not imply that it isn't useful for other purposes. Spam and
message content scanning were not common in 1996!

I also believe that the rationale given in RFC 2033, that the LMTP
protocol should not be used over WANs because it can aggravate the
situation described in RFC 1047, is faulty. While TCP connections do
sometimes fail because of network errors, I believe the most common
reason for duplicate messages is that processing of the message on the
server side takes longer than the timeout on the client side.

Consider the case where a client has a configured timeout of 5 minutes
(that's shorter than required in RFC 2821, but not uncommon) and
processing on the server takes 1 minute initially plus 30 seconds for
each of the ten recipients.

In SMTP, the server will send back a 2xx reply after 6 minutes. At that
time the client has already quit and will retry later. The message will
be duplicated for all recipients (unless the server detects that
connection is already closed).

In LMTP, the server will send the first 2xx reply after 90 seconds, and
another reply every 30 seconds. The client will never come close to the
timeout of 5 minutes. Even if it foolishly doesn't reset the timeout
after receiving each reply, the message will only be duplicated for the
last two or three recipients.

Also, some of the requests and comments have suggested that it doesn't
do enough, such as Peter Holzer's suggestion that the mechanism for
wrapping the list of responses may not be robust enough to avoid
duplicates (and the mechanism I present is already a LOT sturdier than

So the question is whether adding a final response after the per-recipient
resposes helps with this problem. The authors of RFC 2033 seem to have in
mind an implementation that performs each recipient's delivery action then
sends that recipient's final response. We're going to have to be a bit
more sophisticated. In particular, there's a security consideration: you
do not want to re-scan the message for each recipient because this makes
it easier for an attacker to use lots of your CPU. A safer model is to
scan the message once, then different per-recipient policies depend on the
result of that scan (e.g. postmaster is much more lax than other
recipients). This design allows you to send out the final replies as fast
as possible, e.g. using a single TCP send.

I'd rather depend on the TCP stack using Nagle's algorithm to
concatenate the replies that doing that in the server process. I have in
the past sometimes grossly underestimated the maximum time something
could take. So - purely as an advice to server programmers - I'd suggest
that the server process send out any reply as soon as it is available.
If the server sends the replies fast enough the OS will coalesce them
into a single packet anyway.

Holzer's suggestion effectively requires this, because he makes the
server send the overall message status before the per-recipient

No, that was just poor phrasing on my part. "message accepted for some
recipients" should really read "message accepted for zero or more
recipients". It's the individual codes that count.
draft-varshavchik-data-smtpext (which I wasn't aware of) uses a 5xx code
instead of a 2xx code as a "wrapper".

The intention was that the server would send out the initial 250- line
as soon as it has received the data (and possibly performed tests on it
which apply to all recipients) and then send the partial replies as it
performs the tests for each recipient.


   _  | Peter J. Holzer    | I know I'd be respectful of a pirate 
|_|_) | Sysadmin WSR       | with an emu on his shoulder.
| |   | hjp(_at_)hjp(_dot_)at         |
__/   | |    -- Sam in "Freefall"

Attachment: signature.asc
Description: Digital signature