spf-discuss
[Top] [All Lists]

Re: Re: DNS load research

2005-03-21 16:47:05
Andy Bakun wrote:
While, yes, you would have to look up the MX for domains that you would
not normally correspond with when resolving an SPF record that uses an
mx mechanism, this is no different than having to handle increased email
load in general.  The result of which is largely cachable.

Correct me if I'm wrong, but I thought we're fighting spam because we want to decrease the amount of resources wasted as a result of it. SPF only exists because of that desire, so why does it not make sense that the traffic that it itself generates is as low as possibee?

The good point of simplicity and convenience is about the only reason why a _little_ extra traffic can be tolerated. I think this may be a good argument, but I subscribe to the idea within reasonable limits.

Also, MX mechanisms really are worthless, but expensive:

When you list an MX mechanism, there can be two possible scenarios:

1. You control that MX mechanism.
   So you know all the mailers, and you should list them (by IP :) )

As I've said before, the mx mechanism eases maintenance, and if we need
to encourage use of SPF compilers or add code to DNS servers to replace
mx with a list of ip4s at query time, then so be it.  Using arguments
like "you can do easy maintenance with a makefile and macros" is
disingenuous because not all DNS information, even within the same zone,
is maintained by the same person with the same permissions on even the
same system.  There are LDAP and RDBM systems out there that store
enterprise-wide network topology information, and different departments
can be responsible for different records and parts of the tree that
eventually get served by DNS.  Guy recently said it better than I am, I
think.

I don't know how that works, but I'll look into it. Any info you can point me to would be appreciated.

As for the thing I said I'll get to in a minute... looking up MX records
returns the A records of the values of MX in the additional section of
the result, which makes both aol.com and leave-it-to-grace.com MX look
ups only 1 query.  Again, there's nothing to stop a smart DNS server
from populating the additional section of the result of a TXT query for
an SPF record with information to avoid further queries.  If this
existed (and the market might bring this about) the exact cost (in
number of queries) is then different based on the exact version of the
DNS software being run by the SPF serving domain.  Again, a reason to
avoid a total look up count and just compare mechanisms' expenses in
relation to each other.

This is very interesting, it would make sense if aol.com used custom changes to their DNS servers to optimize their load, but do you know if other standard servers include this kind of functionality ?

How much of the internet can do this kind of optimization currently?

If the goal is to have everyone publish SPF, we must deal with that scenario. How much will it cost our DNS infrastructure if everyone published SPF ? Currently only a small percentage do, and looking at the traffic numbers I don't like where they are headed.

The only thing I can suggest to combat this is that SPF not be deployed
at all.  Then our DNS numbers would be great, but our stats for forged
email that no one wants would be in the toilet.  I, for one, would like
to throw money at getting more bandwidth and CPU power so I don't have
to deal with forged email (_your_ mileage may vary).  SPF actually has
the capability to do this, in a predictable, scalable way that gets to
the root of the problem (unlike things like baysian filters and MUA
checks (SenderID) that require the mail to be accepted first).

"Boy, this world-wide-web thing is great, but now my DNS servers are
serving X times as many requests than when I just ran a gopher server."

Andy, I consider you to be one of the more thoughtful participants in this mailing list. Please don't loose it now. :)

The HTTP protocol is wonderfully light on the DNS system, probably as light as the gopher protocol used to be.

The DNS that represents a web server is only queried once for each ISP of a visitor to the site. So if 1000 customers of earthlink.com surfs to www.coolsite.com, the DNS server of earthlink.com will only query the coolsite.com domain ONCE and then serve the response to its customers from its cache. For heavily used sites, there is for the most part one query per day per ISP (assuming TTL=1day). Looking at my own server, which is used very litely, I have on average 10 queries/day for www.ohmi.org, but about 100 web-page hits. And this is DNS if you only count hits, not bytes transfer, this ratio of 10/100 is very high probably, because I have very little traffic, perhaps 10 unique visitors daily who subscribe to different ISPs in the world.

As far as I can tell, the SPF system is the application with the highest DNS load that the DNS system has seen so far. Is there another that compares well to SPF?

I am not comparing current volumes of DNS traffic, because SPF is just getting started. I am comparing the designs of the applications that use DNS. I think SPF is by design much heavier on DNS than anything else we've seen so far. When/if everyone adopts it, it will be the heaviest by volume too.

Regards,
Radu.

-------
Sender Policy Framework: http://spf.pobox.com/
Archives at http://archives.listbox.com/spf-discuss/current/
Read the whitepaper!  http://spf.pobox.com/whitepaper.pdf
To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/?listname=spf-discuss(_at_)v2(_dot_)listbox(_dot_)com

Attachment: radu.vcf
Description: Vcard

<Prev in Thread] Current Thread [Next in Thread>