----- Original Message -----
From: "Rolf E. Sonneveld" <R(_dot_)E(_dot_)Sonneveld(_at_)sonnection(_dot_)nl>
Subject: Re: Bounce/System Notification Address Verification
BUT with 100% without of doubt, no SMTP level false positives that did
not
have a meaningful reason.
Excuse me. On March 11, 2004 your server rejected a message of me with
perfectly legal envelope from address. Part of the message header:
Fair enough. Lets study it. First, lets ask if you report it? If so,
please accept my apology. I don't recall, but if you did, rest assured, it
would of been addressed. Why would it not be address?
Date: Thu, 11 Mar 2004 19:48:02 +0100
From: "Rolf E. Sonneveld"
<r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>
Let me get the log...... last year :-) ahhhhhh, here we go...
Two attempts tried at 13:47:49 EST and 14:35:37 EST, same results.
Lets use the last one:
Client IP: 193.67.246.66 (unknown)
14:35:37 S: 220 winserver.com Wildcat! ESMTP Server v5.7.450.9b13 ready
14:35:37 C: EHLO donald.sonnection.nl
14:35:37 S: 250-winserver.com, Pleased to meet you.
14:35:37 S: 250-SIZE 5120000
14:35:37 S: 250-ETRN
14:35:37 S: 250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
14:35:37 S: 250-AUTH=LOGIN
14:35:37 S: 250 HELP
14:35:39 C: MAIL FROM:<r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>
SIZE=2560
14:35:39 S: 250 <r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>... Sender
validation pending.
Continue.
14:35:39 C: RCPT TO:<winserver(_dot_)support(_at_)winserver(_dot_)com>
14:36:45 ** WCX Process: wcsap ret: 552 (Rejected by WCSAP CBV)
14:36:45 S: 552 Return Path not verifiable.
14:36:46 C: QUIT
14:36:46 S: 221 closing connection
As you can see, there was 1:14 seconds WCX process. Lets see what
happen at the WCX process:
20040311 14:35:39 -------------------------------------
20040311 14:35:39 version : 1.55 / 1.54
20040311 14:35:39 calltype : SMTP
20040311 14:35:39 state : rcpt
20040311 14:35:39 cip : 193.67.246.66
20040311 14:35:39 cdn : donald.sonnection.nl
20040311 14:35:39 from :
<r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>
20040311 14:35:39 rcpt :
<winserver(_dot_)support(_at_)winserver(_dot_)com>
20040311 14:35:39 sapfilter : pass (time:31)
20040311 14:35:39 saprbl : testing 66.246.67.193.sbl.spamhaus.org
20040311 14:35:40 saprbl : testing 66.246.67.193.list.dsbl.org
20040311 14:35:42 saprbl : testing 66.246.67.193.bl.spamcop.net
20040311 14:35:43 saprbl : pass
20040311 14:35:44 sapspf : none (time:1062)
20040311 14:35:44 sapcbv : total mx records: 3
20040311 14:35:44 try mx : mail.sonnection.nl ip: 193.67.246.66
20040311 14:35:44 # connecting to 193.67.246.66
20040311 14:35:45 S: 220 donald.sonnection.nl --
Server ESMTP (PMDF V6.2-X10#39908)
20040311 14:35:45 C: NOOP WCSAP v1.55 Wildcat! Sender
Authentication Protocol
http://www.santronics.com
20040311 14:35:45 S: 250 2.0.0 OK.
20040311 14:35:45 C: HELO mail.winserver.com
20040311 14:35:45 S: 250 donald.sonnection.nl OK, [208.247.131.9].
20040311 14:35:45 C: MAIL FROM: <>
20040311 14:35:46 S: 250 2.5.0 Address Ok.
20040311 14:35:46 C: RCPT TO:
<r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>
20040311 14:36:21 C: QUIT
20040311 14:36:45 sapcbv : -1
20040311 14:36:45 result : reject (0)
20040311 14:36:45 smtp code : 552
20040311 14:36:45 reason : Rejected by WCSAP CBV
20040311 14:36:45 wcsap finish (66125 msecs)
Ok, at 44 seconds, it became the CBV, as you can see thre wasn't a response
to the RCPT TO: The timeout appears to be 35 seconds.
This is version 1.54, over a year old. The default timeout is 35. That has
not changed, but I believe we got 2-3 reports among all thousands customers
processing millions of messages, and I recall in each case, it was zombie
sites, so the short delays helped here but I remember the sysops did say
they would increase.
I believe I stated a few times the CBV is a compliant SMTP-based client with
the exception of the timeouts.
If I remember correctly your CBV rejection had to do with limited
bandwidth on my side (yes, I live in Europe and bandwidth in Europe may
still be an issue for small companies). Your CBV system failed to verify
my mail address (the same address as this very message has), but on that
same day I received 179 messages in my mailbox, associated with that
same mail address. Furthermore my server tried to send the message for
several days, but finally gave up. So the CBV for this message must have
encountered the same problem for several days, in which I was able to
receive hundreds of messages from the Internet.
Apparently there is a major delay at RCPT TO. However, I don't see it as a
bandwidth issue, but rather at your RCPT TO step. May I ask why it is taken
so long here? Is this some sort of ant-haverst throttle?
But I am not convince with your statement makes the CBV a flawed concept.
I suspect the timeout values you use (or used) for your CBV are set too
small to be useable in the real world of the Internet.
I can increase the timeout, but you must understand, it will to address the
exception rather than the rule. The real world DOES NOT have delay RCPT TO
value of 35 seconds or more. That is a fact. I can 100% understand that
it is concern and consideration, but to state the "Real world of the
intenet" I don't agree with. Thats like sayiing all the installations are
just boozo systems leaving in non-real world environments when on the
countrary, hardly any our mail business customers are hobbiest but small to
large ISP and businesses world wide. If it was a problem - guaranteed, it
would of been DUMP along ago. The real world don't have long delays at
RCPT TO.
I reported this false positive to you, but you blamed my system for
being unresponsive.
Ok, I don't remember the conversation. But I trust you are correct and if
that is what I said back then, then I can see what happen. Yes, looking at
the details, it was not a bandwidth issue between your system and ours, but
some unexplainable delay at RCPT TO.
What are the timeout values for your CBV connections, i.e. primary
connection setup timeout, timeout for EHLO/HELO, timeout for MAIL FROM
and timeout for RCPT TO?
As I indicated, 35 seconds is the default. If anything, it is the only
non-compliant back of the CBV. I can increase and I doubt it will solve
your problem.
The question you should ask is why the RCPT TO: was taking so long? It
wasn't the bandwidth.
This brings me back to the topic for which this mailing list does exist:
SMTP. If CBV is one of the possibile ways to fight spam, the use of it
should have been documented in some RFC to prevent all sorts of
interoperability problems.
In the general sense, "Fight Spam" can be used. But from my perspective it
is to validate the Return Path with no consideration to mail content
analysis. The benefit of such is to fight spam simple because the majority
of the abusers do have bad return paths.
Such a document on CBV should contain things
like:
* timeout values for the various CBV commands
Right. Making it per state should be a consideration, but keep in mind,
it would be a fine tuning issue because we need to learn about the
exceptions to the rules such as your long or unresponsive RCPT TO command.
* mail addresses for which a CBV should not be done (only <>, not
postmaster(_at_)domain, not mailer-daemon(_at_)domain etc., or?)
Right. For backward compatiblity, NULL only should be used.
However, we have been in R&D with with CBV system that understand
each other. I can't expand on this yet.
* how to treat domains names (just use MX technology?) to find out
against which host to verify
* what to do if there is no MX record, but there is an A record
(operate the same way as 2821?)
For backward compatibility, it must support full SMTP client MX lookup.
Please keep in mind of the target audience - the BAD people.
* what to do if an MX points to 127.0.0.1
Reject! If your MTA is calling, that is natural IP based authorized session
on must systems.
* what to do if the envelope from address points to one of your own
servers (for small enterprises you can reject such messages, but
for large scale environments you can't do that, think e.g. of
mailing list messages which come back from the Internet with your
domain)
A CBV works well when YOU completely established a anonymous final
destination tractions. Considerations such as your own domains need to be
done before the CBV, but you also need to make sure your own domains are
not spoofed by unknown IPs.
Lets check if you have an SPF record.... No. You should consider adding
SPF support, this will help you protect 100% your local domains.
* how to deal with situations where primary MX host is down,
secondary is up, secondary does not know which address is valid
and which is, outbound MTA suffers from CBV because primary MX is
down)
Again, a full SMTP client MX expansion logic (Thanks to Valdis to help me
get the most of all the consideraitons here.) For a CBV, the only thing
you can only do is issue a temporary negative response code (45x).
Fortunately, again, if it wasn't for the fact that over 60-80% of the
transactions are bad, then the CBV would not have a big payoff.
The good systems will try again, the bad systems will not. Maybe they will
with a different transaction, but they ususally do not. You see it, but
what really funny is to seem some that try different RCPT TO: but with the
same return paths. :-)
So let me suggest you write this document and then send it to ietf-smtp
for review (I think ietf-smtp is appropriate, as CBV seems to use SMTP
technology, at least in part).
I have considered it. But if I haven't said it yet, I think the CBV is a
temporary solution.. It works because it based on backward compliancy. If
we do a new optimal approach, it won't have a payoff because again - the
target market - the bad guy - isn't going to comply with new stuff, until we
force the issue.
So the CBV is a "BLACK BOX" Return Path Validator. The return path is a
requirement which I don't think will every change. So as soon as someone
comes out that is better, it is plug-play.
I have 3 drafts in the works. A few things keep occuring that holds me back.
But with one I have briefly talked about in various places and it is based
on focusing in the SMTP TMS (Transaction Management and Security) of the
state machine with plug and play security modules based on the parameters
that exist.
result = EPV-CONN(IP)
result = EPV-HELO(IP,CDN)
result = EPV-MFROM(IP,CDN, RP)
result = EPV-RCPT(IP,CDN, RP, FP)
result = EPV-DATA(IP,CDN, RP, FP, PL)
we need to fact the fact that in lieu of a complete revamp (which I doubt),
we need to see there will many proposals, many ideas, and the least that the
SMTP people can do is develope a new set of guidelines that puts allows the
interfacing to occur.
Last yeat I outline how this "SMTP CORE" model can be used to define the SPF
model, the SENDERID model, the CSV model as well as CBV. So that joe sysop
can use one method he likes and the other guy use somtehing else, and the
interfacing would be standard.
The beauty of this approach is that the CORE will stay, but new emal
authentication ideas will continue to come along. So we can plug and play
with it too.
That is whats missing in my view.
I hope John's new effort for 2821bis/2822bis is real effort to address the
real issues the real world confronts today - the SMTP exploits that feeds
into the email security problems. But for some reason, I think I might be
getting my hopes too high.
We will never be able to eliminate all the subjective mail content issues.
That will always remain a administrative/user/vendor situation. But I think
we can sincerely address the automated technical aspects of the transport
system and do so with a high degee of success.
If there is one thing that is common - the problem is universal. Everyone
has an interest to address the issue. All we need is a focused direction
and dedicated champions to help make it happen.
As you know, a lot of legitimate mail is being sent with non-replyable,
non-verifiable mail addresses. Let me give you two examples why CBV
would be disastrous for me:
1. a purchase order acknowledgement for schoolbooks I purchased
online for one of my sons; in the message were details about how I
could pay for these books.
I understand. But in my view, this is commercial environment and it would
be prudent of the commericall system to comply with standard SMTP 2821
operations. So they should be a valid return path - not a bad return path.
I acknowledge there could be an exception, but it is not the rule. The
problem is we allow the exception to relax the process, thus given us
loopholes that are exploited.
Personally, when I did similar transactions of important, I made sure I
provided before I make the purchase:
a) Either an email address that would be white listed, or
b) I would temporarily or permannent white list the vendor
I usually do -a- because I don't want the vendor to have a permanent address
that isn't related to our relationship. It also helps see if the vendor is
selling the email address, etc.
Now, I don't expect anyone to do this specific recommmendations and it is
certainly not programmed or documented that this should be done, but it is
all very easily doable with today's software.
Again, if the vendor is serious, supporting SPF will help you authorize
them.
2. an invitation from the Dutch tax administration service for my
company in which I am told to send in a tax declaration before the
end of the month. This electronic declaration is mandatory, if my
server would have rejected that message, I would have had a
serious problem.
Again, a special relationship. I don't recommending using CBV should not be
used when a social/'business relationship is established. You don't need
it.
But in general, for the unsolicited anonymous email from the good person,
the CBV is based on the premise that the Return Path is valid and correct
from good systems, and it usually is.
Why would a good system use a bad address?
Can a good system be broken?
Yes, it is possible. If this was even 5% possibility on a regular basis - I
would dump the CBV, but think about that.
If the CBV is that unreliable because the 5% of all good systems have bad
return paths, then that pretty says that the bounce system is unreliable as
well and the return path is pretty much useless.
Anyway, thanks for your message. I would like to know what happen with the
RCPT TO delay. Will increasing it to 2 minutes do it? I guess I can
afford to do this because you would be the exception, but then again, I can
easily white list you and the issue is mute.
Thanks
--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com