Re: Use of New Mask Mechanism

Radu Hociung wrote:
> David MacQuigg wrote:
>
> > The DNS Load Research thread is getting way to long with too many 
> > topics.  Let's start a new thread to work out the details of how the 
> > new mask mechanism should be used.  Radu has a good start with his 
> > procedural outline below.  I inserted one added condition, based on my 
> > current understanding of DNS, which may be flawed.
>
> Great idea. Thank you! :)
>
> > SPF users should be urged to keep their compiled records simple (list 
> > of IPs only) and shorter than 450 bytes.  Our Best Practices document, 
> > and some good examples of large domains like rr.com, should show that 
> > this is possible.  The advanced mechanisms should be documented in an 
> > appendix, to be used only for the rare cases where they are truly 
> > needed, like an "SPF Obfuscation Contest" :>)
>
> Yes, Andy has volunteered to kick-start the Best Practices documentation 
> based on available sources and ideas we have discussed already (If I 
> understood his offer correctly).
> I would suggest that once he's done a first path, including as much info 
> as he can find, then we all can offer text that has not already been 
> addressed, or improve the existing text.
> I would almost make the 450-byte limit an implementation requirement for 
> compilers. It is already in the draft.
> > We might want to consider a different syntax for the mask mechanism, 
> > considering how difficult it has been to explain, even to the experts 
> > on this list.  I suggest a simple "not" operator, which complements 
> > whatever follows.  -!ip4:65.122.44.0/24 means reject any IP *outside* 
> > of the given block.  This term would be evaluated in the normal 
> > sequence, like any other.  To be useful, the compiler would always 
> > place it at the beginning of the record.
>
> I'm ok with changing the syntax, but it looked to me that it wasn't the 
> name of the *modifer* that caused confusion, but functional differences 
> between it and the existing ip *mechanisms*.
> Also. Whatever syntax we chose, it has to be compatible with the current 
> draft, else existing SPF implementations would break instantly.
> The ! between the prefix and the mechanism is not legal according to 
> current syntax. Also, since we do need a *modifier*, not a *mechanism*, 
> the syntax for the modifier name is alphanumeric characters only, 
> followed by '='. Anything can follow the = except space :
>    modifier         = redirect / explanation / unknown-modifier
>    unknown-modifier = name "=" macro-string
>    name             = ALPHA *( ALPHA / DIGIT / "-" / "_" / "." )
>
>    macro-string     = *( macro-expand / macro-literal )
>    macro-expand     = ( "%{" macro-letter transformer *delimiter "}" )
>                       / "%%" / "%_" / "%-"
>    macro-literal    = %x21-24 / %x26-7E
>                       ; visible characters except "%"
>    macro-letter     = "s" / "l" / "o" / "d" / "i" / "p" / "h" /
>                       "c" / "r" / "t"
>    transformer      = *DIGIT [ "r" ]
>    prefix           = "+" / "-" / "?" / "~"
>
>
> Actually, if I read the above correctly, -m= is not legal. m-= is ok, 
> but m~= and m?= are not. So lets make it
>    m = [PREFIX] [IP notation]
>
> where [IP notation] is defined like in the bind man page (man named.conf):
>
>
>      ip_prefix      An IP network specified in dotted-decimal
>                     form, followed by "/" and then the number of
>                     bits in the netmask.  E.g. 127/8 is the
>                     network 127.0.0.0 with netmask 255.0.0.0.
>                     1.2.3.0/28 is network 1.2.3.0 with netmask
>                     255.255.255.240.
>
>      dotted-decimal One or more integers valued 0 through 255
>                     separated only by dots ("."), such as 123,
>                     45.67 or 89.123.45.67.
>
>
> For IPv6, RFC3513 speficies how the net-blocks should be specified:
>
>
>    "The text representation of IPv6 address prefixes is similar to the
>    way IPv4 addresses prefixes are written in CIDR notation [CIDR].  An
>    IPv6 address prefix is represented by the notation:
>
>       ipv6-address/prefix-length
>
>    where
>
>       ipv6-address    is an IPv6 address in any of the notations listed
>                       in section 2.2.
>
>       prefix-length   is a decimal value specifying how many of the
>                       leftmost contiguous bits of the address comprise
>                       the prefix."
>
>
> ie,
>
>
>    mask-string      = [ prefix ] ip-netblock
>    ip-netblock      = as per bind MAN page for IPv4 or RFC3513 for IPv6
>
>
>
> Once again, the mask would not work as a mechanism (unless it was in 
> include, like Frank mentioned), because each mechanism can return a 
> match. the mask modifiers can return a match only after *all of them* 
> have been checked against the IP. Think of a mask like m=65/6 m=214/6. 
> For senders in the 214 net, your proposed -!ip4 mechanism would 
> wrongfully declare the 214 sources as "FAIL".
> Also, I would revert back to my original syntax that allows m=65/6 
> instead of requiring m=65.0.0.0/6. Both would work of course, but the 
> simpler syntax would be legal. This is because I don't understand why 
> that syntax has to be compatible with other systems. Maybe someone can 
> explain. I think that as long as the checkers recognize the mask 
> modifier, they will also know (from the draft) how to interpret its 
> contents.
>
> > At 12:15 AM 3/26/2005 -0500, Radu wrote:
> >
> > > Let me quote the rules for creating masks first:
> > >
> > > If no compiler
> > >    1. there is no masking, or you'd have to insert it manually,
> > >    which is inconvenient and error prone.
> > >
> > > elseif compiler is used
> > >    2. If compiled with cron, or once in a while, -flatten should
> > >    not be used. There will be left-over mechanisms whose
> > >    resulting IP address may change (administrative gap)
> > >    Thus, masks MUST not be added, since while they work initially
> > >    they would break the record when the ISP changes their
> > >    infrastructure.
> > >    3. If compiled with cron, and -flatten not used, but the record
> > >    compiles into a list of IPs anyway (ie, you list no mechanisms
> > >    that lie outside your adminstrative boundary), then it may
> > >    include masks if useful.
> >
> >
> > Masks should be added to a list of IPs, only if that list is already 
> > too long to fit in one 512-byte DNS response message.  In this case, a 
> > mask may allow the SPF check to return a FAIL without initiating a TCP 
> > connection to retrieve the full DNS message.
>
> The compiler currently has an option to specify the max length of an 
> output SPF string, in case a name server has a lower limit for the 
> length of TXT records.
> So the number of characters allowed by the name server software will 
> dictate the max length of string that the compiler is configured to 
> produce. Since the mask has to be in the first record, it will cause the 
> compiler to shorten the number of bytes used for mechanisms such that 
> the the top record, including the mask and any other modifiers fits 
> within the server's TXT record limit.
> I am afraid it won't be so easy to provide a clear number like "450".
>
> Bind insists on providing the NS records for the zone with every 
> response. The more you have, the less room is available for the TXT 
> record. Also when the domain name is longer, that takes some space away 
> too. Ie, in the response packet of _s4.ohmi.org, the name takes up 13 
> bytes, but for a _s4.longer-domain.name.com, it takes more space, 
> leaving less available for the TXT record.
> It looks to me like it will take some work to figure out how many bytes 
> are available for the TXT record, and what all the variables are.
> At ohmi I use 3 name servers and BIND, and the biggest TXT record I can 
> fit into a 512-byte UDP packet is 357 bytes. YMMV, but it proves that 
> even a seemingly conservative limit (450 or 400) is not always 
> appropriate. If I had more slave name servers or a longer domain name, 
> my usable TXT space would be even less.
> So I suspect that for sites that compile their record with a cron job, 
> they should find a value for the -len parameter that works on their 
> system. Alternately, the compiler should automatically figure it out.
> Do we have any volunteers to take up the task of understanding the DNS 
> response packet and writing up a report, and a formula/algorithm that 
> can be used the maximum TXT record size available at an arbitrary 
> domain? Reading through RFC 1035 may not be enough, and we have to 
> consider some of the more popular DNS servers to see what their quirks 
> are (for instance insisting to provide additional "helpful" records even 
> though doing so requires them to drop to TCP mode).
> This is some real research work that must be done sooner or later.
By the way. Since SPF2/PRA records share the same packet space with the 
SPF1, we will definately have problems. One of these two MUST move to 
their own hostname. For instance _spf.{domain_name} or _spf2.{domain_name}.
Since there are more spf1 records than spf2 records currently published, 
it will probably be PRA that must move.
The move could be done in a backwards compatible way:

If the {domainname} TXT query did not return a PRA record, but did 
return an SPF1 record, try looking at _spf2.{domain_name}
else
There is no PRA record, don't do the extra query.

We should bring this up to the PRA people before they get too close to 
the point of no-changes-acceptable.

> > >    4. Masks can only be reliably inserted when your record is
> > >    completely in your administrative control, as above, or if
> > >    the compiler runs as part of the DNS server.
> > >    5. In that case, the record can be safely flattened *only if*
> > >    the TTLs of all mechanisms are respected,
> > >    including those across the domain boundary. In that case, the
> > >    record, and implicitly the mask get regenerated every time
> > >    the IP list gets regenerated beause of expired TTLs. So,
> > >    the mask always reflects the current record.
> > >
> > >    Also, there's an additional condition on inserting masks.
> > >    6. A mask may only be inserted if all mechanisms that cannot be
> > >    compiled into an ip list (those that use the %{l} or %{i} macros)
> > >    are brought up into the top TXT record.
> > >    In other words, a mask may only be inserted if the remaining
> > >    mechanisms in subsequent redirects/includes contain only
> > >    IP lists.
> > > end if.