From: Roger Moser
Sent: Thursday, June 10, 2004 1:25 AM
Stuart D. Gathman wrote:
Just include both SUBMITTER and MAIL FROM in the exists for the
MAIL FROM.
The code in the senders DNS server then does the same check as the
receiver would. What is not happening now is checking with MAIL FROM -
only the SUBMITTER is SPF checked.
And there was an excellent suggestion to make that MAIL FROM check
a CBV check, but using DNS instead of SMTP - just lookup a name
containing
the relevent info.
I have a similar request as the original request in this thread:
Suppose all mail from my mail server has a signed envelope sender
and I have
set up a custom DNS server to check the signatures by using the 'exists'
mechanism. And I want to prevent that a spammer uses SUBMITTER
(or similar)
with "v=spf1 all" to send spam "from my domain". So my SPF record
would say
for example:
"v=spf1 only=orig exists:%{S}.ses.example.com -all".
The "only=orig" modifier would mean "check only the SPF record of the
original domain, and ignore any SPF record of the domain of the SUBMITTER
or the source route (which could be a spammer)".
Following would have to be added to the specifications:
"The SPF implementation MUST first check the SPF record of the original
domain, if known. If it contains the modifier "only=orig", then the SPF
implementation MUST ignore the SPF record of the SUBMITTER (or similar)."
I can see the utility of this and this is a reasonable way to get it. Since
a number of us are interesting in validating the originating address, and
that adds a _lot_ of value to the SPF protocol, let me review a few ways to
do that and see what ultimately makes the most sense. What this adds to SPF
is not only more confidence that the 2821 originating address is not a
forgery, but having a validated MAIL FROM: address makes it safe to postpone
2822 checks until after the end of DATA, since we know it is safe to send a
DSN. Regular SPF checks on a forwarded message _do not_ provide that
assurance. While I am no fan of XML, since the 2822 tests can be postponed
until after the end of DATA, this also allows a full XML parser and a TCP
method of fetching the XML record, should people decide that is how they
want to go. While that's not my preference, many people have expressed
interest in that approach.
Here are some ideas on how to get SPF to do originating address validation,
and I would like to put them down as a starting point for discussion.
Constructive criticism is welcomed and encouraged. This topic is bound to
raise some people's blood pressure so I hope we can stick to the technical
merits and avoid the sarcasm and personal insults that have appeared in
other threads. In the interest of conducting a civil discussion, I suggest
that other list members simply _not respond_ to any overly sarcastic or
abusive posts. There is simply no excuse for lack of civility and we don't
have to tolerate it. Decide for yourselves, no one is in charge here but
you.
possible modifications to SPF syntax
------------------------------------
1) Domains need a way to express the fact that all outgoing mail bears and
SES or SRS signature and the recipient should produce a given SPF result in
the absence of one. Pro: lets a domain owner warn recipients to make sure
the originating address bears an appropriate signature and what SPF result
they want to assign when a signature is absent. Con: adds a mechanism,
modifier or exists macro to the language definition, requires an additional
DNS query to be useful. Overall: helps close one of the biggest holes in
SPF and improves its ability to detect forgeries before DATA.
2) Domains need a way to express the fact that their DNS servers can
validate plain email addresses, SES or SRS signed addresses, want the
recipient to query for validation and produce a given result on failure (or
have the DNS server return the SPF result). Pro: lets a domain owner
require that the recipient validate the purported originating address via
DNS to generate an SPF result; all the validation overhead is borne by the
sender. Con: adds a mechanism, modifier or exists macro to the language
definition; requires two additional DNS queries to be useful. Overall:
provides a mechanism to close a hole in SPF even more completely than #1
above and improves its ability to detect forgeries before DATA; gives user
highest confidence that the originating address is good; makes it safe to
delay additional 2822 checks after the end of DATA.
3) Roger's original suggestion: Domains _might_ want a way to express the
fact that they want the recipient to validate the originating address and to
not do SPF checks on the forwarder. This is a sensible optimization, sense
since if the originating address is correct, why should anyone care if the
forwarder has their SPF configured correctly or not? Validating the
originating address converts SPF into an end-to-end verification system and
this makes SPF checks on the forwarder redundant. As such, it appears to be
an optimization of #2 above. Pro: lets a domain owner tell the recipient
that they should not bother to check SPF on the forwarding address, if one
exists; saves one or more DNS queries and an SPF evaluation in the case of
forwarded messages where the originating domain owner has elected this
option; if enough people did this, we wouldn't need a sender rewriting
scheme of any kind. Con: adds a mechanism, modifier or exists macro to the
language definition. Overall: there is probably more benefit than cost and
this would be a useful addition.
4) Domains _might_ want a way to express the fact that their DNS servers can
validate plain email addresses, SES or SRS signed addresses and let the
recipient decide if they want to do the additional query. I think this is
not a good idea. SPF evaluations should be deterministic and always give
the same result regardless of the recipient's local policy. The recipient
must decide _what_ to do with a given SPF result through its local policy,
but the SPF result must be the same regardless of who evaluates it. Pro:
allows recipients the option of not validating originating addresses if they
are lazy. Con: allows recipients the option of not validating originating
addresses if they are lazy. Overall: a bad idea, since SPF results must be
deterministic.
In terms of SPF syntax, my conclusion is that we should make provision _now_
for originating address validation and that Roger's suggestion is a good
optimization. I therefore suggest that we figure out some way to express #2
and #3 in SPF syntax and try to incorporate them into the standard before it
is too late.
SPF operation
-------------
1) Whenever there is an SES signature, or there is an SRS signature and the
originating and forwarding domains are the same, do a query for the
originating domain and see if the domain requires validation of the
originating address. This would work in the case that there was a
recognizable signature in place, but all a spammer would have to do to
defeat it is to leave off the originating signature. Therefore, this mode
of operation would generally do checks on valid signatures and catch only
the dumbest spammers. Pro: catches dumb spammers who try to forge an
originating signature. Con: most of the queries would be for valid
originating addresses, easy to circumvent by leaving off originating
signature when forging. Overall: to little benefit for too much cost.
2) If the current sending domain SPF check passes, always query the
originating domain to see if the domain owner has expressed any policy
concerning originating addresses. If there is policy, implement it. If
there isn't, you are done. This would definitely work, but would require an
extra DNS query on every forwarded message. Since the majority of messages
are probably not forwarded (anyone have any statistics on this?), the
overall impact of this depends on what percentage of incoming messages are
from a forwarder on a given MTA. The plus side of this is that if the
originating domain owner has published a policy on originating addresses,
you stand a better chance of rejecting forgeries before DATA. Because of
this, the extra overhead of the extra DNS queries would be worth it. A
significant shortcut is available here for most messages: if the current
sender is the originating domain, this check has already been done! Pro:
always determines when the originating domain has policy about originating
addresses and if they provide a validation mechanism, extra queries to
determine this only required when the incoming message is from a forwarder.
Con: must do an extra DNS query for all incoming messages that are from a
forwarder and some will not have any additional policy to implement.
Overall: this is bullet-proof but has some wasted overhead in the case of
forwarded messages with originating domains that don't publish an
originating address policy.
3) Check the originating domain first to see if they have requested that no
SPF checks be performed on forwarders. Perform an SPF check on the
originating address, if the domain owner published such a policy. If that
passes, and there is a forwarder, and there is no prohibition against
checking SPF on the forwarder, do the SPF check on the forwarder. This is
an optimization of #2 above and can save DNS queries and SPF checks in the
case where the originating domain owner provides validation means for
originating addresses. This turns SPF from a hop-by-hop verification to an
end-to-end verification, therefore, it is unnecessary to check the
credentials of intermediate hops. Pro: always determines when the
originating domain has policy about originating addresses and if they
provide a validation mechanism, avoids extra DNS queries and SPF evaluations
on forwarded messages when the originating domain publishes address
validation policy and directs the recipients to not bother to check SPF on
forwarders. Con: adds an extra DNS query that provides no information if
the originating domain owner does not publish address validation policy,
significant change to current SPF operation, although there are no
interoperability problems with previous versions. Overall: as bullet-proof
as #2 above, but lower overhead in some cases.
4) Put an extra ESMTP parameter in MAIL FROM: that indicates if there is
originating address policy. This is functionally very similar to #1 above
and shares its properties. Pro: catches dumb spammers who try to forge an
originating signature, avoids wasted DNS query if originating domain doesn't
publish and originating address policy. Con: most of the queries would be
for valid originating addresses, easy to circumvent by leaving off the
additional ESMTP parameter when forging. Overall: too little benefit for
too much cost.
In terms of SPF operation, my conclusion is that it is important enough to
provide for originating address validation to modify existing SPF operation
to accommodate it. I therefore suggest we find a way to incorporate #3
above before it is too late.
--
Seth Goodman