Re: Abort data transfer?

Another way to look at this and probably how it can manifest itself is 
with prolonged DATA processing on the server side, enough so that the 
client drops the line and retries.
This is indirectly noted in RFC 5321 section 6.1 with:

   To avoid receiving duplicate messages as the result of timeouts, a
   receiver-SMTP MUST seek to minimize the time required to respond to
   the final <CRLF>.<CRLF> end of data indicator.  See RFC 1047 [40]
   for a discussion of this problem.

The reference RFC 1047 (Duplicate Message and SMTP) describes:

   Many mailers delay responding to the final dot because they are
   doing sophisticated processing of the message, in an attempt to
   confirm that they can deliver the message.  For example,
   the mailers may expand an entire mailing list to confirm that
   it can reach all addressees or may attempt to physically
   deposit the message into the mailboxes of local users,
   before confirming receipt of the final dot.  These practices
   are not unreasonable, but they often cause the synchronization
   gap to continue for several minutes, and increase the
   likelihood that the sending mailer will timeout or the network
   will fail before the accepting 250 reply is sent.

   AVOIDING SYNCHRONIZATION PROBLEMS

   The best way to avoid the synchronization problem is to minimize
   the length of the synchronization gap.  In other words,
   receiving mailers should acknowledge the final dot as soon
   as possible and do more complex processing of the message later.

However, the issues we found.

The 1988 RFC 1047 note was molded in the days of single threaded, 
process, slower machines and lower bandwidth. The relative rare (less 
volume) malicious accept/bounce mail attacks were not taken as 
serious. The mindset was that a "user" attempting to fake mail should 
not degrade or disabling the benefits of a "relaxed" open email 
system.  This mindset is shown in RFC 5321 section 7.1.
The problem of course is today's reality:  Its no longer a "user" but 
an multi-billion dollar problem and industry of users trying to fake 
mail.  With multi-threaded designs, faster machines, better 
scalability and efficiency makes it all possible for more SMTP 
receivers to do processing dynamically at the SMTP DATA level.
Nonetheless, if a server data processing is prolonged, we found there 
are clients who drop at 3 minutes after the <CRLF>.<CRLF> and before 
the server issues a reply code.
Because of the SMTP NO QUIT CANCEL SMTP semantics, the server detects 
the drop and may cancel the transaction. Systems following the RFC 
1047 guidelines, always accept and post the message are subject to 
duplicates.
So the lesson is probably that the DATA processing, if any, needs to 
be fast and less than 2-3 minutes.
--
Sincerely

Hector Santos
http://www.santronics.com


Sabahattin Gucukoglu wrote:

> On 20 Oct 2009, at 02:20, David MacQuigg wrote:
> > Arnt Gulbrandsen wrote (about letting the transmitter hang):
> > > Some people do this in order to be as occupy the sender's resources. 
> > > There's a fancy name for it. You send one byte every few minutes, so 
> > > it takes three hours to send the whole SMTP response. I don't know 
> > > whether doing so really inconveniences the $#(_at_)$#@! spammers and other 
> > > evil creatures who initiate most SMTP connections.
> > I wasn't thinking of a tarpit.  That would involve keeping a 
> > connection on my end.  Just dropping the connection will leave the 
> > transmitter with an SMTP process filling memory for maybe ten minutes.
> Really?  I rather fancy it will inform the sender's TCP that you're done 
> with it, and the next few failed sends will go completely unnoticed by 
> ratware and actioned upon by connection closure by a real MTA on a 
> failed send or on an error detected on read by some more fancy 
> event-driven process.  In any case the length of the long wait is one 
> TCP buffer, or not very long at all except perhaps for sneakernet. :-)
> Perhaps you mean something more complex, maybe losing TCP state 
> completely at your end.  That might make a few long-suffering 
> implementations wait a good while to give up.  But if you were going to 
> go that far, you might as well turn a real BOFH, advertise an MSS of 1 
> to evil clients, drop every packet except the fifteenth a turn, and 
> delay each acknowledgement by five minutes. >:->
> > SM wrote:
> > > Section 4.5.3.1.7 of RFC 5321 discusses about the maximum total 
> > > length of a message content.  There isn't a permissible way within 
> > > SMTP to abort during DATA.  It's a "don't do it unless you know what 
> > > you are doing".
> > I see no technical problem on the server side with sending "552 Too 
> > much mail data," then a TCP reset.  If the client keeps retrying, 
> > without fixing the problem, I would block that IP address for a few 
> > hours.
> It should work, but you can't depend on it.  It would work with my 
> client since all errors are detected during reads and not writes.  But 
> you simply can't depend on a client maintaining state the way you hope 
> it should.  If sends fail because you've reset the connection, you may 
> simply cause the sender to abort the attempt and retry.  It's that simple.
> > Sabahattin Gucukoglu wrote:
> > > My implementations note the reason for rejection, then switch the 
> > > event handler for the connection to one that just reads each buffer 
> > > and then throws it away.  There is nothing more that can be done.
> > This could leave your receiver tied up for as long as the client keeps 
> > spewing garbage.
> Correct.  SMTP is not "Broken", as some people would have it, but it's a 
> cruel world out there.  Be prepared for the worst.
> Though I don't say you should not try to defend against DoS attacks, 
> realistically any malicious person already has numerous ways to attack 
> available services no matter how you protect them.  For SMTP, rate 
> limit, connection limit per IP, etc are about the best you can hope 
> for.  A session timeout may also be plausible, though I didn't implement 
> one in the server, only RFC-specified inactivity timeouts and am only 
> half-convinced of a need to put it into the client.
> > > Even trying to inform the sender by writing the error to it while it 
> > > is sending won't necessarily work, because it might not choose to 
> > > look at its inbuffers until it's quite finished spewing.  And in any 
> > > case that's standards violation territory you're wandering into.
> > Which standard?  The paragraph above from Section 3.3. of 5321 seems 
> > to be talking about after the 354 reply is issued (past tense).  I 
> > assume that means any time after that, not necessarily waiting for the 
> > end of data.
> Nothing in RFC 5321, as far as I know and understand it, allows you to 
> circumvent the lock-step nature of SMTP.  That section seems to 
> corroborate that.  The best you can do is suddenly become unavailable, 
> with apologies to the client.  I suppose you can interpret the text to 
> mean that closing the connection for other reasons is acceptable but as 
> has already been explained by others this isn't called for for any case 
> other than resource unavailability and will, quite reasonably, give you 
> client retries.
> > > So you'll just have to do what you can to keep the connection alive 
> > > for the data to come in, then reject it and hopefully at that point 
> > > you can resume the dialogue to finish up and ensure the sender will 
> > > leave you alone.  Anything else is basically suboptimal.
> > If we have a broken or malicious sender, it seems that the optimal 
> > solution would be one with the least resources on the server side.  
> > That would be not even keeping the connection alive.
> > That works only if you don't want retries.  So it's suboptimal.  
> > Maybe, as has been said before, we've simply never been bothered 
> > enough about it to put it right, and make it more optimal.  But right 
> > now, accepting data is basically mandatory, except when it isn't, 
> > which is to say, at the beginning of the response to the DATA command 
> > or after the final dot, and not inbetween.
> Cheers,
> Sabahattin