[Top] [All Lists]

Re: Mail Data termination

2011-08-16 22:40:34


I fully agree it is a state machine which generally means there is little room for subjective opinions. I'm writing here in general and not to specifically to you, you know all this.

As a background. RFC821 had a lower case "should not" for closing the connection before issuing a QUIT, where RFC2821 (carried on to RFC5321) has a upper case MUST NOT mandate of requiring a QUIT. The different between the two are significant.

I am not in a position to take one side only. We have to support both modes. Some (very few) customers want their SMTP server to behave the old way and most want to enforce SMTP compliancy.

First, there are plenty of clients who even after seeing a 250 will not interpret this as an indicator the message transaction is complete. Until they are able to enter the final state and issue QUIT (or a state where it moves into new transaction command), they will continue to retry. The only ones I have seen that stood out that use the 250 as the only requirement are older sender software using the 821 model. They will not resend.

Second, spammer exploitations of what they assume are continued support for relaxed 821 legacy operations has allowed them to blast commands and data with no respect to server responses. So you see many transactions where the residue unread text being now read at after DATA in the SMTP command state.

Because of these residue text and redundant consecutive 500 response, the server limit kicks in and drops the connection. If the server used the 250 message accepted as a trigger to deliver the mail, the spammer found a loophole.

Third, it is written, a QUIT is a MUST mandate (stated as a MUST NOT drop connection until a QUIT is issued). Not enforcing this mandate, returns the server back to 821 mode and the exploitation problems.

Forth, the only reason the issue was highlighted was because it was false positive by a IETF mailing list I am subscribed too and I took the time to find out why it happen.

In short, it was technically an invalid RFC5322 message or probably more correct, the SMTP client failed to properly escape dotted lines for whatever reason (I think Ned is correct that it probably did the QP conversion on the fly).

So the only highly subjective argument which I will never buy as being absolute in either direction is that the receiver MUST ALWAYS deliver with the 250 even for invalid messages.

What is lost here is that the sender was broken and the truncated message was indeed no longer invalid.

Also consider a typical philosophical conflict is the idea of SMTP level filter vs POST SMTP filtering where perhaps a 250 is always issued and delivery is pending the post smtp processing result after the session has been ended. There is no guarantee of delivery.

If we use the old 821 idea that a 250 is all that is needed by a client, then QUIT is not a requirement. Client can just drop.

But that is not what I see with many senders. The transaction is not complete until the SMTP mandate QUIT an be issued. I only see the old 821 behavior with older software and I say that because they don't retry when server drops the line due to all the 500 responses.

Which finally, either mode is not a problem until an non-compliant operation take place.

John C Klensin wrote:

--On Tuesday, August 16, 2011 20:20 -0400 Hector Santos
<hsantos(_at_)santronics(_dot_)com> wrote:

OK, I don't think it is productive to repeat these semantic
difference of opinions of what section says:

Hector, having either written those words or inherited them from
Jon Postel and RFC 821, maybe I should offer an opinion.  It is
really important to remember that SMTP is stateful (see 2.3.6
among other things) and that the DATA command:

        -- Is treated as atomic wrt that state model, with the
        354 message being a special "go ahead" indicator, not a
        command termination acknowledgement
        -- Terminates the mail transaction in progress.

Mail transactions are initiated by a MAIL command (see Section
3.3) and extend until a QUIT or RSET is issued or a DATA command
completes, whichever comes first.   I think the spec is very
clear about that -- if you disagree, point out where and,
preferably, supply text.
Note (also in Section 3.3):

        The end of mail data indicator also confirms the mail
        transaction and tells the SMTP server to now process the
        stored recipients and mail data.  If accepted, the SMTP
        server returns a "250 OK" reply.

Note that it does not say "tentatively confirms pending the next
command" or any such thing.  The server gets EOD (CRLF.CRLF),
treats it as confirming the transaction from the client's
standpoint, initiates processing of the message, and treats the
mail transaction as ended.
IMTO, no matter how you wish read this, NO QUIT means a
cancellation because otherwise the only other way to end a
SMTP session is for the client to close the socket connection.

QUIT immediately after the DATA command (and hence the mail
transaction) completes is just a signal to the server that the
client is initiating an orderly close.  It has no effect on the
mail transaction because there isn't one open.  When the server
receives it, the SMTP session is closed.    QUIT (instead of
DATA) after one or more RCPT commands is another matter, but
that isn't the case we are discussing.

Similarly, having the client close the connection after EOD is
sent is, indeed, "a cancellation", but it is a cancellation of
the SMTP session... it has nothing to do with any mail
transactions.  On the other hand, if the client closes the
connection prior to sending EOD, that inevitably cancels both
the mail transaction and the SMTP session... having the server
deliver anything would be a really bad idea and is clearly (I
believe) outside  the spec.

So what this would indirectly suggest the SMTP specs
implicitly says:

    There is two ways for a client to end a SMTP session:

That is true even though one of those choices is prohibited
unless the "operational necessity" provisions are invoked.

and that does not jive with the written words:

    The sender MUST NOT intentionally close the transmission
    channel until it sends a QUIT command,

It aligns perfectly well.  A client that closes the transmission
channel without first sending a QUIT is in violation of the
spec.  That doesn't mean it can't do it.  Because it can do it
and, more important, because all sorts of other things can cause
a connection to be dropped, a server has to be prepared to deal
with a dropped connection and to clean up when it occurs.

I just don't see a problem there.  Please point out what I'm

 Aside: there is a design characteristic here that I wish
        were different in SMTP.  I have wished it for any years,
        that has done me no good, and I can see no possible way
        to change it now, at least at a plausible level of
        effort.  SMTP does not anticipate a case in which a
        client opens a mail transaction, sends RCPT commands as
        needed, sends DATA and then changes its mind and wants
        to abort sending without having the message delivered.
        With SMTP as now defined (and as it has always been
        defined), it can do that _only_ by closing the
        connection prior to sending EOD.  While the use cases
        are few, that is just unsanitary, especially in a model
        in which we expect closing of an SMTP session to be
        preceded by a client-server handshake (i.e., sending
        QUIT).   In an alternate universe, we might have had
        EOD-and-deliver and EOD-and-abort, not just the first.
        But we are getting real close to 30 years after the time
to make that choice differently.
I can understood your position, very clearly, but in general,
I don't think this is a good way to do robust client/server
communications (allowing drops to signal message transaction
completion). It opens the door for unsureness.

But, as I hope is clear from the above, no such thing happens.
If the drop occurs after EOD, the mail (message) transaction is
over already, completed by EOD.  If the drop occurs before EOD,
the mail transaction and the SMTP session are abandoned and,
because EOD wasn't received by the server, nothing is delivered.

There is a potential problem with that model, but it is
connected to EOD in progress, not to the situation you are
apparently concerned about.  That issue is discussed in great
detail in RFC 1047.



Hector Santos

<Prev in Thread] Current Thread [Next in Thread>