Re: Queued Mail or Unreturnable Mail?

Sabahattin Gucukoglu wrote:
> Hi all,
>
> This question is bothering me.  It's stalled my reading of 2821bis, and the 
> discussion around the issue (having diverged from IPv6) seems worthwhile 
> fodder for continued debate.
> I've got a message in my queue here destined for natwest.co.uk.  
> Natwest.co.uk exists, but it isn't running an SMTP service; it's intended 
> as a web server.  There's no MX, and I don't check SPF.  (Both natural 
> conditions for typical hosts.)  The message is a DSN generated by this 
> host.  The original message was intended for a primary host for which I am 
> an acting secondary.  I will only accept mail for primaries when I can't 
> connect to them myself, with a few exceptions to help routing difficulties 
> of some hosts reaching the primary directly.
> Which is the problem?
If I followed you message right, this is a normal undeliverable message.

Your receiver accepted the payload first, then determined it was not 
deliverable, your system properly created a bounce.
This is normal other than you now fall into the ACCEPT/BOUNCE attack 
dilemma many in the world fall into.
If your receiver was able to dynamically to determine the acceptability 
of the RCPT TO forwarding path and issue a rejection at the SMTP level, 
then you would not have this BOUNCE issue.
> 1.  I am naive enough to receive mail from non-verifiable senders. 
Yes. <g>

If this is a issue for you, then you need to take that extra step.  Some 
systems don't for many reasons, including scalability reasons and for 
many, there only option is to do a RCPT TO verification after the 
payload was accepted and SMTP session has ended.
Many old school systems run like this, including ours. But we long 
copied the same post gateway hosting verification logic to the SMTP 
process and this was the beginning of addressing the abusive 
ACCEPT/BOUNCE problem.  I remember the many customers calling "I am 
getting killed with this bounces" and they didn't even know they had the 
ability to do the same POST checking at the SMTP level dynamically. 
That was because they were so use to running like the old days, 
UUCP/SLIP, with a separate Mail Gateway that did all the hosting 
verification it was never an issue until it became abusive.
> I have 
> not gone even to the extent of checking that the domain exists; a surefire 
> but, as you can see here, pointless check.  
Right, that is why we have a CBV implementation. At the very least, you 
need to try to connect to the system to see if its even has a SMTP 
listening server port.
Also, your bounce retries can have special rules.  Like for example, in 
our implicit MX logic, we have a "1 try" concept and this serves us well 
to eliminate the junk.
> I will never be happy until 
> every conceivable check is done to verify that, should an error occur, I 
> wil *always* be able to send mail back to the sending site at the sending 
> mailbox.  
I have the same philosophy. IMV, if a client is able to send you mail, 
then it must have a companion host system available to receive, 
especially at same precise moment of time.  Low chance it will claim 
"Receiver Host is temporarily down" without the whole system not being down.
But overall,  SMTP is about two communications.  SMTP is not about send 
only outbound broadcasting system without having some contact address. 
The system is designed for feedback.
> 2.  I am not doing enough to ensure that I will not have to generate a DSN.  
Right, this goes back to considering using a DYNAMIC SMTP level RCPT 
verification.
And if possible, do no checking until RCPT TO is acceptable.  This is 
the recommendation in 2821.
We have statistics showing this will not only drastically reduce your 
bounce issues, but it will drastically reduce any necessary verification 
overhead by nearly 65-70%. Of course, that is site dependent.  We get 
hit a lot with tons of old Compuserve addresses and also I think an old 
employee sold our support system user list to spammers when it was an 
open email system back in the 90s.   But that was turned off and we 
still get hit by all the bulk spam. IOW, at least 65% of rcpt to state 
fail making it prudent that we don't do any MAIL FROM checking (like SPF 
or CBV) until RCPT TO is known.  A massive improvement across the board, 
especially in DNS lookups.
> My vote is for 2 on general grounds of practicality, ease 
> of implementation and least harm done.

Both.

Once upon a time, such filtering was 100% left to the operator. From an 
ethical standpoint, I felt the duty of SMTP (any mail protocol) was to 
follow the rules as much as it can and transport mail with no questions ask.
But as the SPAM problem became too much, and began to directly to effect 
us, then it was time to begin "doing more" to help our customers.
The first thing we added was CBV, then SPF and a p-code filtering 
language to add rules at each state of the SMTP state machine.
Our general AVS logic policy is strong SMTP compliancy at all states, 
except DATA where we make no subjective rules based on mail content. 
Operators make those spam filtering rules. Not us.  We just provide the 
hook into the DATA state and push strongly that they all rejection be 
done at SMTP level.
--
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com