[Top] [All Lists]

CBV systems - was Re: SMTP Extensions - proper peply code for disabled commands

2004-01-13 14:58:47

----- Original Message ----- 
From: "Keith Moore" <moore(_at_)cs(_dot_)utk(_dot_)edu>
To: "Hector Santos" <winserver(_dot_)support(_at_)winserver(_dot_)com>
Cc: <moore(_at_)cs(_dot_)utk(_dot_)edu>; <ietf-smtp(_at_)imc(_dot_)org>
Sent: Tuesday, January 13, 2004 12:22 PM
Subject: Re: SMTP Extensions - proper peply code for disabled commands

I assume that this CBV service uses SMTP to determine whether the
return-path is valid.

Your presumption is correct.

hmmmmmm,  Maybe I should plead the 5th?  :-)

If so, does that CBV send MAIL FROM:<>?

Configurable.  However, a NULL address is highly suggested for the reason
you suspect.

If so, what happens when the SMTP server associated with the return-path
(incorrectly) returns an
error to that? (in my experience this is a fairly common bug, or
misfeature, or misconfiguration).

In our testing with over 25 commerical systems, millions of transactions.  I
believe I have personally seen logs with maybe a few of these type of
sessions. In all cases, it was confirmed to be "zombie-like" sites that
intentionally fail at the protocol level.

In our system,  a presumption of a non-compliant remote system is made with
a less than 100% trust value provided to the overall result.   Since I have
a pretty good idea where you are going with this, I should note a common
non-understanding of CBV systems (atleast for our system) is that it focuses
on looking for 100% true negatives (return paths rejected).  Little or no
trust is provided to those systems that accept the return path, hence NO
decision is made.    However, in our system,  SMTP compliancy it inherently
part the total process.   I made it a SYSTEM POLICY issue that the system
admin can choose.   Emperically, we have no found any problems (false
negatives) and when there were, their filter tables were quickly adjusted.

The bottom line is that it works in testing verifible BAD addresses.  Not
GOOD addresses because no trust can be put on this type of result.  This is
a KEY focus I wish to make to any one who may be philosophically against a
CBV.  I understand very clearly be a heated debate, especially for many who
have their mind already made up.  But most of the arguments I have seen is
that you can trust an accepted return path.  That is true.   The trust is
the rejected return path.

Other issues of OVERHEAD and REDUNDANCY was left to the optimization phase
of the project. They were also addressed.

If the CBV doesn't send MAIL FROM:<>, what happens when the SMTP server
associated with the return-path server also uses a CBV?

Then the second system will perform a CBV as well.   An overhead or
redundancy issue that should be address with the SMTP protocol.

I think the more appropriate question is:

What is the RFC 2821 presumption for the validity of the return path?

Or at what point should RFC 2821 presume the return path to be valid?  When
it is provided or when its too late (after the mail is received and

These are some of the things I would like to discuss in the near future when
I finish collecting my thoughts in how to best address or discuss the issues
regarding anonymous access exploitation or abuse of the SMTP protocol with
pragmatic gurus of the industry.   A realistic problem that will need to be
addressed sooner than later.

Like I said, I thought about pleading the 5th here because I understand very
clearly the sensitive nature of this topic.  I have much to say, yet it may
not be all worth it the stress that it may generate (especially for those
who are philosophically oppose, right or wrong).   I will leave you with
this in the hope it summarizes most of my thoughts:

1) Anonymous access abuse is the issue, not authorized or authenticated

2) Both theoretically and empirically, the CBV has proved to work to provide
100% true negatives (return path rejected at the remote system).  100% trust
can be put behind this result.

3) Less than 100% trust can be provided to positive results (return path
accepted at the remote system).

4) Implementations can impose other test to measure the trust value of the
positive result.  In our case, we offer a open relay test option.  "Test to
see if ANY obfuscated non-local return path is also accepted."

5) Overhead and redundancy can be reduced a number of ways.  Primarily,
minimize the calls to when they are 100% necessary.  Domain/IP pairs called
DIPs are used as part of an automatic learning process.   The most important
overhead reduction is to call CBV iff a valid local recipient is provided.
In other words, allow the client to reach the RCPT TO state.  If your system
is going to refuse the RCPT TO address, then they is no need to call CBV.
We found that nearly 60% of the mail transaction is refused due to
non-existing recipients.  Hence that is a 60% CBV call reduction in a given

6)  We also use LMAP-based DNS lookup proposals, specifically the DMP
proposal to help validate an authorized sender machine prior to using the
CBV.  This proved to be one of the troubling aspects I had with the ASRG key
focus of using DNS lookup solutions to address the anonymous access problem.
LMAP-based solutions only work for local domain validation.  It can not be
used to validate non-participating remote systems due to high DNS lookup
delays with spoofed domains.

7) Finally, empirical results with over 25 sites processing millions of
messages have shown overwhelming high success.

I can provide more detail information if you like.

Our system also works as a web service.  If you would to test it for
yourself, click the following URL. I will temporary open access to the web
service for the next few days:

Statistics can be found at:

The way I view it is the following:

1) The CAN-SPAM Act has provided a mandate with two SMTP level

- Valid Return Paths
- Advertisement Tags

The marketing of CAN-SPAM Ready Systems (on both sides) has already begun.
At a minimum, this will place product requirements to provide auditing and

2) Mr. Crocker's ASRG Proposal Guideline Document,
draft-crocker-spam-techconsider-02.txt, emphasizes incremental and backward
compatibility minimizes or quelling any desire to fundamentally alter the
SMTP protocol.

I agree.

Hence, in other to satisfy both requirements, the only logical conclusion is
the validation of the return path using the current SMTP standard protocol.
However, the advertisement tag requirement presents a data collection
requirement with the standard protocol. This conflicts with dynamic return
path validation unless it is done at the delayed to the data collection
level as well.

In short, there are some fundamental conflicts with the RFC 2821
specification, the SMTP protocol and the mandate imposed by the CAN-SPAM
act.  Adjustments or rather "clarifications" need to be made in the RFC.

How that is done is the matter left up to discussion.

In the mean time,  our primary or main focus was to re-engineer our software
to provide programmable hooks at each state of the protocol state machine.
See the diagram at

We choose to use  CBV concept as a validation sequence at the RCPT TO simply
because there is NO other way to do it.  We optimized it using intelligent
learning/caching and also implemented RBL and the ASRG DMP DNS Lookup
proposal to reduce the need to call the CBV.

What remains in our view is:

1) Removing SMTP from the fuzzy interpretation of mail content, focusing on
access control,
2) How can two current SMTP systems best communicate to perform this
3) Clarification of the RFC 2821 validity of the return path and WHEN it is

Maybe I should of taken the 5th?  <g>

Hector Santos, Santronics Software, Inc.

<Prev in Thread] Current Thread [Next in Thread>