[Top] [All Lists]

Re: CBV systems - was Re: SMTP Extensions - proper peply code for disabled commands

2004-01-14 07:47:20

I assume that this CBV service uses SMTP to determine whether the
return-path is valid.

Your presumption is correct.

hmmmmmm,  Maybe I should plead the 5th?  :-)

If so, does that CBV send MAIL FROM:<>?

Configurable. However, a NULL address is highly suggested for the reason
you suspect.

If so, what happens when the SMTP server associated with the return-path
(incorrectly) returns an
error to that? (in my experience this is a fairly common bug, or
misfeature, or misconfiguration).

In our testing with over 25 commerical systems, millions of transactions. I
believe I have personally seen logs with maybe a few of these type of
sessions. In all cases, it was confirmed to be "zombie-like" sites that
intentionally fail at the protocol level.

The reason I asked is that I used CBV for many years to filter spam from my mailing lists, and I often found that (otherwise) legitimate addresses failed to verify if I used MAIL FROM:<>. (I was using CBV on the header From address - on the assumption that if someone on the list couldn't reply to the message by normal means, then it probably wasn't useful to send that non-subscriber's comments to the list anyway).

Most of the time the verification would fail in the MAIL FROM response, but occasionally it would fail in RCPT TO response. So I ended up having to send MAIL FROM:<postmaster(_at_)my(_dot_)site> or some such instead in order to get an accurate verification.

(one could argue that you shoudn't accept mail from misconfigured systems anyway, but bouncing such messages doesn't do anything useful IMHO because it doesn't report an error that is likely to be understood by the person receiving the bounced mail. That and you have to arrange for the bounces to have a return-path other than <> in order for them to even get back to the sender.)

If the CBV doesn't send MAIL FROM:<>, what happens when the SMTP server
associated with the return-path server also uses a CBV?

Then the second system will perform a CBV as well.   An overhead or
redundancy issue that should be address with the SMTP protocol.

Here's what I see happening if this is widely adopted:

A0->B0: MAIL FROM:<my.address>
-B0->A1: HELO
-B0->A1: MAIL FROM:<dummy(_dot_)address(_at_)B>
--A1->B1: HELO
--A1->B1: MAIL FROM:<dummy(_dot_)address(_at_)A>
---B1->A2: HELO
---B1->A2:MAIL FROM:<dummy(_dot_)address(_at_)B>
----A2->B2: HELO
----A2->B2: MAIL FROM:<dummy(_dot_)address(_at_)A>

You get the idea. Note that the various An don't all have to be the same hosts; they're just hosts acting on behalf of domain A (A1..An are MXes for A). Similarly for B0..Bn. So it's not necessarily easy for either of these domains to "break the loop" by realizing that they're being asked to verify a dummy return address that is used to verify a real address.

I think the more appropriate question is:

What is the RFC 2821 presumption for the validity of the return path?

Or at what point should RFC 2821 presume the return path to be valid? When
it is provided or when its too late (after the mail is received and

Return-path is intended for (non)delivery notification, not for sender verification. It is not reasonable to take return-path as an indicator of sender identity.

SMTP servers _should_ return immediate error conditions (in response to MAIL FROM, RCPT TO, DATA) whenever possible because, for a wide variety of reasons, immediate reporting is much more reliable and consumes less resources than bouncing the message. For similar reasons, SMTP senders _should_ minimize the relay path length to the receiver's MX rather than aggregating traffic through upstream relays. Of course there are reasons to have both outbound and inbound relays, but these do come at a cost.

Despite my use of CBV on mailing list traffic, I don't think they are nearly as good an indicator for traffic in general, for lots of reasons:

- The presumption that traffic with an invalid return-path is invalid, while perhaps reasonable for some mailing lists, is less reasonable as a general rule. - The verification has lots of overhead, and some potential for looping if widely adopted. - The effectiveness of CBV has decreased considerably over time. When I first started using it (not sure when, maybe 7 years ago) it identified nearly 100% of spam sent to my lists. When I stopped using it on a large scale (just over a year ago) it only identified about 25% of the spam sent to my lists. It appeared that spammers had realized that mail from invalid addresses was less successful, so they simply started sending mail from valid addresses. - In my experience, CBV does not provide 100% true negatives - because of misconfigured systems that reject MAIL FROM:<> and other valid mail. (if you have your SMTP server report 4xx whenever the CBV tempfails, there are other problems - it is not unusual for a CBV to fail because either the sender's DNS servers or the sender's SMTP server are too slow or inaccessible, even though the sender's system is sending outgoing mail.)

In short - CBV is expensive for the receiver and easily defeated by spammers. If widely adopted it very quickly becomes useless overhead.

I do think we need reliable traceability to the sender, but doing CBV on return-path is not the way to get it.

The way I view it is the following:

1) The CAN-SPAM Act has provided a mandate with two SMTP level

- Valid Return Paths
- Advertisement Tags

The marketing of CAN-SPAM Ready Systems (on both sides) has already begun. At a minimum, this will place product requirements to provide auditing and

CAN-SPAM is a crock. We should not presume that the US Congress is qualified to dictate technical standards - heck, they can't even write nontechnical laws that are in the public interest.

2) Mr. Crocker's ASRG Proposal Guideline Document,
draft-crocker-spam-techconsider-02.txt, emphasizes incremental and backward compatibility minimizes or quelling any desire to fundamentally alter the
SMTP protocol.

I haven't read this document yet, so I can't comment on the extent to which I think the argument in the document is valid. But using return-path as a sender identity conflicts with its purpose in SMTP as the destination of (non)delivery reports. The Sender field from RFC 822 is the closest thing to a verifiable sender identity from the original design, but it is so widely misused today that it is not salvageable. Basically the requirements for verification are such that we're going to need a new field.


<Prev in Thread] Current Thread [Next in Thread>