On Jan 5, 2011, at 4:00 PM, Chris Lewis wrote:
On 1/5/2011 2:41 AM, Steve Atkins wrote:
On Jan 4, 2011, at 10:44 PM, John Levine wrote:
With that said, I still like my b-tree hack, which makes queries that
shouldn't get either NOERROR or NXDOMAIN, a lot better as a way to
publish ranges of addresses in a DNSxL.
If you're going to the effort of adding that much functionality
at the client end, switching to a better matched protocol instead
might be better. All the high traffic DNSBL users already use
a push protocol of sorts. Moving to a better, more standardized
one might be a win. A local server for protocol X could still offer
a DNS interface to the MTA to ease implementation
I must confess to a bit of confusion. We seem to be talking about really
naive IPv6 DNSBL implementations blowing out caches on high volume, and yet,
at the same time, we acknowledge that the large DNSBL users already download
zone files.
IPv6 is more likely to blow out caches than IPv4, I think. That's going to
increase the load an end-site places on a central server quite significantly.
And _that_ is going to mean that an IPv6 based list may end up with more
"large" DNSBL users than current IPv4 based systems, as the same end-site might
generate significantly more IPv6 queries than IPv4.
Or maybe it won't. Heck, I'm expecting there to be very little IPv6 mail,
legitimate or otherwise, for quite a long time.
Rsync is essentially the defacto standard for bulk DNSBL transfer, and as you
say it's "not awful". So, we don't seem to have a significant difficulty
with that.
AIUI, rsync is reasonably expensive to offer (in terms of CPU load), and it
adds significant latency to listing and delisting (as you say below). Do you
have any numbers on system load vs users vs update rate conveniently to hand?
I do know of one (commercial) blacklisting system that does use a more
explicitly "incremental" distribution mechanism, but it's probably not that
much better than rsync, and in fact it may be worse.
Google push a large chunk of their entire domain blacklist to every end-user
browser (using a mix of incremental updates, broad blocks pushed to the
browser, with queries to a central server for more specific checks when there's
a broad block match, explicit invalidation of locally stored entries and so
on). It's not the same problem being solved as IP based blacklist distribution,
but it's something that gives some good ideas.
I don't think that looking at an alternate WAN distribution protocol is
something we *have* to do, but if we're looking at changing behaviour at the
MTA location anyway, now's a good time to think about whether we do want to do
that or not, and what the tradeoffs might be. We're a research group, are we
not? :)
Even if we were to do something as simplistic as chop IPv6 queries at the
/64, given that the number of spammers and bots doesn't magically go up
simply because there's more bits to hide in, the caching problem appears to
not that much worse than it already is with IPv4.
The number of spammers may not go up, but the number of addresses they use may.
Or may not, it depends on their behaviour. But I'm not betting that it won't.
IOW, some of the discussion threads here seem to be solutions looking for
problems.
Clearly things are going to shift somewhat. But, it doesn't look like the
real future is much more than:
1) Some mechanism for CBL/XBL single-IP DNSBLs to remain useful (eg:
hardcoded /64 truncation or some mechanism like John's) for Internet query
from small sites.
I think this bit is really tricky, and has lots of possibilities for
over-thinking the problem. Blocking by /64 seems both a good idea, and probably
good enough for small sites.
2) Zone download (Rsync or perhaps something better) becoming more prevalent.
Yup. I think zone download is going to become more prevalent (to the extent
that I think even rsync is going to get too painful to offer in some cases, and
something that looks more like IXFR is worth another look).
3) DNSBL operators will be more conscious of query load and will more
forcefully terminate abusers.
Perhaps we might do more work in (2), to specify zone formats for download.
While the tradeoff volumes for query versus zone downloads/incrementals may
well shift, it will just about be never advantageous for small sites doing a
few dozen emails per day to take a whole zone of something as big as the XBL.
No, it wouldn't. (Then again, there's no such thing as a site doing a few dozen
emails a day, I don't believe. The spam alone is going to be a couple of orders
of magnitude more than that :) ).
Besides, in many cases, that introduces latency delays.
It only introduces latency delays because rsync is not a good protocol for it -
it's not dreadful, but it's not great.
Cheers,
Steve
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg