Re: CBV systems - was Re: SMTP Extensions - proper peply code for disabl

> > I assume that this CBV service uses SMTP to determine whether the
> > return-path is valid.
> Your presumption is correct.
>
> hmmmmmm,  Maybe I should plead the 5th?  :-)
>
> > If so, does that CBV send MAIL FROM:<>?
> Configurable.  However, a NULL address is highly suggested for the 
> reason
> you suspect.
>
> > If so, what happens when the SMTP server associated with the 
> > return-path
> (incorrectly) returns an
> > error to that? (in my experience this is a fairly common bug, or
> > misfeature, or misconfiguration).
> In our testing with over 25 commerical systems, millions of 
> transactions.  I
> believe I have personally seen logs with maybe a few of these type of
> sessions. In all cases, it was confirmed to be "zombie-like" sites that
> intentionally fail at the protocol level.
The reason I asked is that I used CBV for many years to filter spam 
from my mailing lists, and I often found that (otherwise) legitimate 
addresses failed to verify if I used MAIL FROM:<>.  (I was using CBV on 
the header From address - on the assumption that if someone on the list 
couldn't reply to the message by normal means, then it probably wasn't 
useful to send that non-subscriber's comments to the list anyway).
Most of the time the verification would fail in the MAIL FROM response, 
but occasionally it would fail in RCPT TO response.  So I ended up 
having to send MAIL FROM:<postmaster(_at_)my(_dot_)site> or some such instead in 
order to get an accurate verification.
(one could argue that you shoudn't accept mail from misconfigured 
systems anyway, but bouncing such messages doesn't do anything useful 
IMHO because it doesn't report an error that is likely to be understood 
by the person receiving the bounced mail.  That and you have to arrange 
for the bounces to have a return-path other than <> in order for them 
to even get back to the sender.)
> > If the CBV doesn't send MAIL FROM:<>, what happens when the SMTP 
> > server
> > associated with the return-path server also uses a CBV?
> Then the second system will perform a CBV as well.   An overhead or
> redundancy issue that should be address with the SMTP protocol.
Here's what I see happening if this is widely adopted:

A0->B0: MAIL FROM:<my.address>
-B0->A1: HELO
-B0->A1: MAIL FROM:<dummy(_dot_)address(_at_)B>
--A1->B1: HELO
--A1->B1: MAIL FROM:<dummy(_dot_)address(_at_)A>
---B1->A2: HELO
---B1->A2:MAIL FROM:<dummy(_dot_)address(_at_)B>
----A2->B2: HELO
----A2->B2: MAIL FROM:<dummy(_dot_)address(_at_)A>
...

You get the idea.  Note that the various An don't all have to be the 
same hosts; they're just hosts acting on behalf of domain A (A1..An are 
MXes for A).  Similarly for B0..Bn.  So it's not necessarily easy for 
either of these domains to "break the loop" by realizing that they're 
being asked to verify a dummy return address that is used to verify a 
real address.
> I think the more appropriate question is:
>
> What is the RFC 2821 presumption for the validity of the return path?
>
> Or at what point should RFC 2821 presume the return path to be valid?  
> When
> it is provided or when its too late (after the mail is received and
> rejected)?
Return-path is intended for (non)delivery notification, not for sender 
verification.  It is not reasonable to take return-path as an indicator 
of sender identity.
SMTP servers _should_ return immediate error conditions (in response to 
MAIL FROM, RCPT TO, DATA) whenever possible because, for a wide variety 
of reasons, immediate reporting is much more reliable and consumes less 
resources than bouncing the message.  For similar reasons, SMTP senders 
_should_ minimize the relay path length to the receiver's MX rather 
than aggregating traffic through upstream relays.  Of course there are 
reasons to have both outbound and inbound relays, but these do come at 
a cost.
Despite my use of CBV on mailing list traffic, I don't think they are 
nearly as good an indicator for traffic in general, for lots of 
reasons:
- The presumption that traffic with an invalid return-path is invalid, 
while perhaps reasonable for some mailing lists, is less reasonable as 
a general rule.
- The verification has lots of overhead, and some potential for looping 
if widely adopted.
- The effectiveness of CBV has decreased considerably over time.  When 
I first started using it (not sure when, maybe 7 years ago) it 
identified nearly 100% of spam sent to my lists.  When I stopped using 
it on a large scale (just over a year ago) it only identified about 25% 
of the spam sent to my lists.  It appeared that spammers had realized 
that mail from invalid addresses was less successful, so they simply 
started sending mail from valid addresses.
- In my experience, CBV does not provide 100% true negatives - because 
of misconfigured systems that reject MAIL FROM:<> and other valid mail. 
 (if you have your SMTP server report 4xx whenever the CBV tempfails, 
there are other problems - it is not unusual for a CBV to fail because 
either the sender's DNS servers or the sender's SMTP server are too 
slow or inaccessible, even though the sender's system is sending 
outgoing mail.)
In short - CBV is expensive for the receiver and easily defeated by 
spammers.  If widely adopted it very quickly becomes useless overhead.
I do think we need reliable traceability to the sender, but doing CBV 
on return-path is not the way to get it.
> The way I view it is the following:
>
> 1) The CAN-SPAM Act has provided a mandate with two SMTP level
> "requirements:"
>
> - Valid Return Paths
> - Advertisement Tags
>
> The marketing of CAN-SPAM Ready Systems (on both sides) has already 
> begun.
> At a minimum, this will place product requirements to provide auditing 
> and
> tracking.
CAN-SPAM is a crock.  We should not presume that the US Congress is 
qualified to dictate technical standards - heck, they can't even write 
nontechnical laws that are in the public interest.
> 2) Mr. Crocker's ASRG Proposal Guideline Document,
> draft-crocker-spam-techconsider-02.txt, emphasizes incremental and 
> backward
> compatibility minimizes or quelling any desire to fundamentally alter 
> the
> SMTP protocol.
I haven't read this document yet, so I can't comment on the extent to 
which I think the argument in the document is valid.  But using 
return-path as a sender identity conflicts with its purpose in SMTP as 
the destination of (non)delivery reports.  The Sender field from RFC 
822 is the closest thing to a verifiable sender identity from the 
original design, but it is so widely misused today that it is not 
salvageable. Basically the requirements for verification are such that 
we're going to need a new field.
Keith