Re: Use of New Mask Mechanism

At 04:06 PM 3/26/2005 -0500, Radu wrote:
> David MacQuigg wrote:
> > At 02:45 PM 3/26/2005 -0500, Radu wrote:
> > > David MacQuigg wrote:
> > > > At 01:33 PM 3/26/2005 -0500, Radu wrote:
> > > > > David MacQuigg wrote:
> > > > > > At 11:53 AM 3/26/2005 -0500, Radu wrote:
> > > > > > > Once again, the mask would not work as a mechanism (unless it was in 
> > > > > > > include, like Frank mentioned), because each mechanism can return a 
> > > > > > > match. the mask modifiers can return a match only after *all of 
> > > > > > > them* have been checked against the IP. Think of a mask like m=65/6 
> > > > > > > m=214/6. For senders in the 214 net, your proposed -!ip4 mechanism 
> > > > > > > would wrongfully declare the 214 sources as "FAIL".
> > > > > > Oops.  I thought I understood these masks, but I missed it.  OK, so 
> > > > > > what this "mask" mechanism really says is, the IP address must match 
> > > > > > one or another mask range AND match at least one of the subsequent mechanisms.
> > > > > Almost :)
> > > > >
> > > > > Please, let's never call it a mechanism again, to avoid confusion. It 
> > > > > really is a *modifier* !!! Actually a *set of modifiers* are only 
> > > > > meaningful together. Individually, each mask modifier doesn't mean 
> > > > > anything, because it doesn't tell enough to allow the checker to stop 
> > > > > evaluating.
> > > > Good point.  It's a modifier, not a mechanism.
> > > >
> > > > > And what it really says is: Somewhere in the included/redirected 
> > > > > records, there are more IP mechanisms that match some of the IPs in 
> > > > > the mask range. It's a "summary" of the remaining records, if you 
> > > > > wish. The summary includes more IPs than the records themselves, but 
> > > > > it serves well to tell authoritatively what IPs *aren't* in the 
> > > > > subsequent includes. It also serves to tell you what the all at the 
> > > > > very end of the record chain says the to do with unmatching IPs 
> > > > > ("fail", "softfail", etc), so that you don't need to scan the whole 
> > > > > chain to find out what the domain owner wants you to do.
> > > > > Somehow I/we need to find a description of this that would be very 
> > > > > clear, so that implementers of SPF checkers know what to do. It is 
> > > > > clearly a description/language problem because you're not alone 
> > > > > getting a grasp on its meaning.
> > > > How about
> > > >    record           = version [mask] terms *SP
> > > >    version          = "v=spf1"
> > > >    mask             = *( 1*SP m= ipblock )
> > > > After processing the mask, if there is any match, proceed with the 
> > > > terms as usual.  If there is no match, skip processing any terms except 
> > > > 'all', and don't call for any remainder of a truncated record.
> > > Thats good, except the earliest the mask can be evaluated is after the 
> > > A/MX/exists mechanisms. For instance, an exists:{i}.domain.com cannot 
> > > be  covered by a mask. The only reason why a compiler might leave 
> > > a/mx/exists mechanisms in the compiled record is if they contain macros 
> > > that cannot be expanded at compile time (such as i, l, and possibly s).
> > > I was planning on worrying about truncated records. I was going to 
> > > assume that the checker cannot receive a truncated record, because the 
> > > resolver library takes care of using TCP as necessary. In any case, for 
> > > compiled records this should only happen if the compiler screwed up and 
> > > created a record that is cannot fit in a UDP packet. I would treat that 
> > > as a bug that must be fixed, as opposed to a condition that must be 
> > > handled by the checkers.
> > > So, since I'm assuming that the checker receives the entire record, the 
> > > masks could be evaluated when the first include/redirect mechanism is 
> > > encountered. This would mean that the mask does not need to cover IP 
> > > addresses in the top record, as they are free to check.
> > Now I'm confused.  If the reason for masks is *not* to avoid sending 
> > multiple packets, and *only* to avoid processing mechanisms that require 
> > another lookup, why do we need these lookups on the client side?  Why 
> > can't the compiler do whatever lookups the client would do, and make the 
> > clients job as simple as possible?
> Sorry for creating confusion.
>
> Say that you have a policy that compiles to 1500 bytes.
>
> The compiler will split it into 4 records, about 400-bytes each or so.
>
> example.com     IN TXT \
>      "v=spf1 exists:{i}.{d} ip4:... redirect=_s1.{d2} m=-65/8 m=24/8"
> _s1.example.com IN TXT "v=spf1 ip4:.... .... ....  redirect=_s2.{d2}"
> _s2.example.com IN TXT "v=spf1 ip4:.... .... ....  redirect=_s3.{d2}"
> _s3.example.com IN TXT "v=spf1 ip4:.... .... ....  -all"
>
> We want the mask to be applied after the exists:{i}.{d}. Since that
> mechanism was in the initial query, cannot be expanded to a list of IPs
> the mask cannot possibly apply to it.
I think what you are saying is that the compiler can't get this down to a 
simple list of IPs, because we need redirects containing macros that depend 
on information only the client has.  So if we are to put the burden of 
complex SPF evaluations on the server side, where it belongs, seems like we 
have to pass all the necessary information to the server in the initial 
query.  We already pass the domain name.  Adding the IP address should not 
be a big burden, and it would have some other benefits we discussed.
I'm not sure this would entirely remove the threat of an SPF-doom 
attack.  The client still has to wait for the server to complete all the 
processing and return the final compiled record.  A large number of such 
servers could tie up one client without giving the servers much incentive 
to change their evil ways.
My preference would be a tight limit on the number of lookups, and maybe a 
lower priority for processing these lookups.  Simple queries get top 
priority, and lookups are processed if the server is not overloaded.
Maybe I'm just not seeing the necessity of setups like the above 
example.com.  I'm sure someone could come up with a scenario where it would 
be real nice if all SPF checkers could run a Perl script embedded in an SPF 
record, but we have to ask, is that really necessary to verify a domain name?
If we simply can't sell SPF without all these whiz-bang features, I would 
say put it *all* on the server side.  All the client should have to do is 
ask - "Hey <domain> is this <ip> OK?"  We dropped that idea because it 
doesn't allow caching on the client side, but with a simple PASS/FAIL 
response, the cost of no caching is only one UDP round trip per 
email.  This seems like small change compared to worries about runaway 
redirects, malicious macros, etc.
How about this:  All SPF records SHOULD be compiled down to a list of 
IPs.  If you need more than that, then do as much as you like, but give the 
client a simple PASS or FAIL.  Most domains will then say "Here is our list 
of IPs.  Don't ask again for X hours."  Only a few will say "Our policy is 
so complex, you can't possibly understand it.  Send us every IP you want 
checked."
I need to get back to designing ICs. :>)

-- Dave
************************************************************     *
* David MacQuigg, PhD      email:  dmquigg-spf at yahoo.com      *  *
* IC Design Engineer            phone:  USA 520-721-4583      *  *  *
* Analog Design Methodologies                                 *  *  *
*                                   9320 East Mikelyn Lane     * * *
* VRS Consulting, P.C.              Tucson, Arizona 85710        *
************************************************************     *