ietf-asrg
[Top] [All Lists]

Re: [Asrg] How well do DNSxL lookups cache ?

2011-01-04 22:06:22

On Jan 4, 2011, at 7:56 PM, John R. Levine wrote:

Over on the Spamassassin users' list, we've been having a spirited debate 
about the design of DNSBLs and WLs for IPv6, starting with my draft.

One of the key questions is how DNSxL lookups will play with DNS caches and 
servers, which of course depends on the pattern of IP addresses that send 
mail.  David Skoll collected a few stats from his servers, which suggested 
that for low volume mail servers, only about 20% of lookups are answered from 
the cache, and for a largish server handling 1M connections/day, it was only 
about 2/3.

That seemed hard to believe, so I took a week of logs from my mail server, 
did a similar analysis, and by golly, he's right.  The analysis takes 
timestamps and IP addresses, and computes how many DNSxL queries would be 
answered from a DNS cache under various scenarios.  Each group shows the 
number of cache hits and total queries by 24 hour period. (The first and last 
are short, since my logs didn't start and end at midnight.)  Then it shows 
the number of unique IPs queried, and a list showing cache effectiveness, 
with the number of times each cache entry was used, e.g., in the first 
dataset, there was one cache entry used 514 times, down to 536,009 entries 
used only once.

To see how sensitive these answers were to DNSxL design, I tried it with a 15 
minute and 60 minute TTL, and I tried with /24 rather than /32 granularity.  
Neither made much difference: the cache effectiveness rarely got above 20% 
and the vast majority of cache entries were never reused.

Assuming an infinite cache, that'll get much better as the traffic gets higher, 
but it'll still be pretty ugly.

So now I'm scratching my head.  If these numbers are typical, I don't 
understand why DNSBL servers don't all fall over now.

Because high-volume DNSBL consumers don't use them.

They use a local copy of the database, pushed out to them (typically via rsync, 
which is not ideal, but not *dreadful*) and they only use DNS as a LAN protocol 
to communicate between their MTA and their local authoritative DNS server, 
dedicated to serving DNSBL data.

Moving away from running DNS for DNSBLs across the public internet has been an 
obvious move for a long time. IPv6 does move us further in that direction.

DNS is still a perfectly good local protocol, though.

Cheers,
  Steve

_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg

<Prev in Thread] Current Thread [Next in Thread>