----- Original Message -----
From: "Rolf E. Sonneveld" <r(_dot_)e(_dot_)sonneveld(_at_)sonnection(_dot_)nl>
To: "Hector Santos" <winserver(_dot_)support(_at_)winserver(_dot_)com>
Cc: <ietf-smtp(_at_)imc(_dot_)org>
Sent: Wednesday, January 14, 2004 5:40 PM
Subject: Re: CBV systems - was Re: SMTP Extensions - proper peply code for
disabled commands
Read Keith' answer again: he states: 'it is not reasonable to take
Return-path as an indicator of _sender_ identity'. And you are talking
about Return-path as the place to send notifications to. These are two
different things:
1. a mail address to send notifications to
2. a sender's identity
See also RFC2821:
Hi Rolf,
hmmmmm? I hope I did not express the idea that I was referring to the
"author address?"
The author of the message plays no role in the CBV process. The Sender or
"messenger" expressed with the Mail From: return path is the entity involved
here.
So the Return-Path may change on the way to the recipient and is assumed
to be set by the MTA which does the final delivery. This shows you that
chances are high that the Return-Path header will not contain the
original Sender address and as such cannot be used for sender
verification purposes.
Hmmmmm, unless I am wrong, I was under the current/modern SMTP design
assumption the initiating "Message Sender" must remain constant throughout
the entire multi-host route and that the Return-Path: header is only added
during final delivery. This has to be true in order for the final
delivery to stamp the proper Return Path header with the proper return path.
Am I wrong?
As a side note, am I correct in saying the mix terminology of "message
sender" vs "message originator" as used here in '2821' section 4.4 para 6,
is somewhat conflictive?
4.4 Trace Information
...
It is possible for the mailbox in the return path to be different
from the actual sender's mailbox, for example, if error responses are
to be delivered to a special error handling mailbox rather than to
the message sender. When mailing lists are involved, this
arrangement is common and useful as a means of directing errors to
the list maintainer rather than the message originator.
I believe in technical discussions with others, "message sender" is agreed
to be the MTA (SMTP-client) transporting the message. Although most people
clearly understand the difference and the intent of the paragraph, it might
help if the paragraph is cleaned up a bit. Is this correct? Am I still
confusing something?
Further, this paragraph:
The primary purpose of the Return-path is to designate the address to
which messages indicating non-delivery or other mail system failures
are to be sent. For this to be unambiguous, exactly one return path
SHOULD be present when the message is delivered. Systems using RFC
822 syntax with non-SMTP transports SHOULD designate an unambiguous
address, associated with the transport envelope, to which error
reports (e.g., non-delivery messages) should be sent.
I believe the statement clearly presumes that an unambiguous return-path
MUST (not SHOULD) be valid in order for the delivery of a"system failure"
message can be attempted. The idea an attempt will be made can only suggest
that the presumption is one of validity. Whether it is realistically valid
or not is not the question, the presumption is, that is it valid in order
for the attempt to take place. Otherwise, what the primary purpose of the
return-path? It's value is reduced.
AFAIK MS Exchange does verify the syntax of an address during SMTP
transmission, but _not_ the existence of the recipient address.
I believe this is a matter of implementation as it is possible to hook into
the RCPT TO: state.
The number of Exchange servers, connected directly to the Internet is
growing :-( As a side note: due to dictionary attacks mail system
managers will configure their mailservers (as far as the MTA supports
it) to always give a meaningless answer to the RCPT TO: command.
Well, IMO, I believe it is a red herring in the spammer world. I believe
spammers are more interested in reaching the data state and sending the
message regardless of what happens from that point. They have no control of
what happens once the data is received. If we are to believe AOL.COM's
claim as the largest database of email users, then why is it they do dynamic
validation of the recipient address? Why don't they delay the process? I
believe they have concluded it will make the problem worst by lowering the
value of their end-user addresses making it possible for fraudulent usage.
In any case, IMO, a system not validating local recipient address helps
SPAMMERS, not deter them in any way and it adds more burden and cost to the
rest of the industry by given them a database of spoofable addresses.
I contacted them for this issue (as they provide their
customers with an option to violate the standards) and they were not
willing to change this configuration option. From my postmaster duties I
know that 1-2 percent of all external mail systems run IMail. And I've
seen other mailers rejecting NULL envelope From addresses.
Let me do a quick stats count. This is from our own logs since
November 23, 2003 (the start of our
CBV project):
Statistics: November, 2003 day 26 to 30
wcsap calls : 6834
skips : 4204
-bad domain : 323
-no mx records : 0 (old log did not record mx total)
-bad connects : 583 (includes errors on multiple mx)
smtp sessions : 1981
-refused welcome : 12
-refused noop : 0
-refused helo : 1
-refused mail from : 129 ( 6.51%)
-refused rcpt to : 1060 (53.51%)
Statistics: December, 2003 day 1 to 31
wcsap calls : 47782
skips : 19615
-bad domain : 2808
-no mx records : 829
-bad connects : 6749
smtp sessions : 20125
-refused welcome : 350
-refused noop : 1012
-refused helo : 5
-refused mail from : 885 ( 4.40%)
-refused rcpt to : 15374 (76.39%)
Statistics: January, 2004 day 1 to 14
wcsap calls : 4972
skips : 3864
-bad domain : 218
-no mx records : 103
-bad connects : 247
smtp sessions : 843
-refused welcome : 11
-refused noop : 49
-refused helo : 0
-refused mail from : 25 ( 2.97%)
-refused rcpt to : 785 (93.12%)
The refused mail from was more than I thought, but still negligible
overall. These are null addresses. The skips are other preliminary checks
before the actual CBV is done. But more importantly look on how high that
that RCPT TO
rejection is! This is something that can not be ignored. We have a
proof of concept. :-)
FYI: the current SPF draft claims to be a superset of RMX and DMP. See
http://spf.pobox.com/intro.html. I agree with you that SPF has its flaws.
SPF reduces the DNS lookups to one as oppose to one or two lookups (and
possibly more with a DMP=ALTDOMAIN alternative domain response) with the DMP
proposal. However, they all suffer from non-compliant systems. The
process residence time increased tremendously with high DNS lookup delays.
So what I did was provide a table of DMP domains the admin can setup for
these special lookups (local domains and known compliant systems) for now.
It helps to eliminate the local domain spoofing quite nicely.
The main problem in my view is that everybody tries to find his/her
solution to the SPAM problem, proposing all sorts of extensions to SMTP
or to the way MTA's handle mail, but there is no single coordinated task
force taking responsibility to start a _fundamental_ discussion about
how to solve/minimize this problem. Many proposed solutions are
solutions for an ideal world and don't take into account the real
messaging world of today (with a myriad of possible usages of e-mail).
E.g. the SPF folks assume that everybody will change the way they are
doing mail forwarding.
I agree with you 100%.
In any case, although the problem is complex, I believe the scope can be
reduced to a technical SMTP level. I truly believe that the SPAM problem is
just a symptom with the real problem that we need to deal with, anonymous
access.
Technically, I believe SMTP protocol is sound. Developers need stronger
functional specs in order to better enforce anonymous access controls.
Yes, the forefathers will say "the internet is built on relaxed
restrictions. you can't enforce anything." This should not take away the
idea that we should clarify what is expected in the functional
specifications.
If the HELO can be VALIDATED, then we should say it MAY be validated and a
transaction can be stopped for non-compliance. We shouldn't say "you must
not reject mail for failed validations." We need to clearly define what
machine domain must be.
HELO must a FQDN and a domain literal for dynamic IP machines.
It MUST match the connection IP address.
Once stated, responsible developers will comply. Commercial User agents
will change their wares to comply.
The same applies to the MAIL FROM: return path.
Either its valid or its not. It can't be both ways. Whether we agree with
CAN-SPAM or not, it is here. That is a reality. Valid return paths are part
of the its mandate. At a minimum RFC '3821' should be made consistent with
what the law now expects and enforces on spammers, whether they support it
or not. All I am saying by this is that the RFC should make it very
possible that new future implementations CAN and MAY enforce strong
restrictions based on validating machines and the return path at the
protocol level. As it is right now, the loop holes exist in the functional
specification.
We close the loopholes in the functional specifications and we will begin to
immediately minimize the anonymous access
problem. A status quo position will make it fundamentally difficult for any
solution to work. Efforts will move in different directions.
How it is done?
Well, I suggest for backward compatibility that we begin by using extended
EHLO attributes and commands to help define the level of SMTP conformity of
the sender and receiver. i.e;
C: EHLO machine-domain
S: 250-winserver.com, Pleased to meet you.
S: 250-RFC3821
S: 250-SIZE 5120000
S: 250-ETRN
S: 250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
S: 250-AUTH=LOGIN
S: 250 HELP
Possibly, RFC3821 can also be a tag in the greeting.
220 xxxxxx, ESMTP-RFC3821 [product info] server ready.
Either way, this extended attribute begins to inform the client (regardless
if they are reading it or not) that the new RFC3821 policies are in effect
for the session. RFC3821 offers stronger anonymous access controls a
implementation may impose consistent with the intentions of the
specifications with no ambiguity.
Of course, implementations have design it anyway they like - relaxed or
strong for non-compliant clients. But my main point is that we give the
power back to the developer to stop the abuse.
I have an idea that is based on using the MX record to define the SMTP level
of conformity. Why not use a subdomain naming syntax, i.e., (note, for
illustration purposes only)
mx1-SMTP3.example.com
Or, lets use the preference value + 1000 to signify a SMTP3 system.
Whatever is done, the goal is to give current systems with little change in
their software a immediate and fast method to provide information expressing
the level of conformance the target host may have without any extra lookups.
In any case, currently I think any solution that is sought are all weaken by
the fact the RFC is fundamentally inconsistent with the efforts being made.
This thread pretty much proves my point. We seem to be stuck on a time
warp. Anonymous access is a problem that must be addressed sooner than
later. :-) As far as the CBV is concern, the unfortunate part about it
is that it works all too well from a proof of concept point of view in
eliminating 80% or so the spoofers. Whether CBV is the method of choose is
not the issue. If something better arrives, it is be used. But CBV has
high-lighted that by using SMTP conformity, we can eliminate a tremendous
amount of the abuse in the mail transport system.
--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com