ietf-asrg
[Top] [All Lists]

Re: [Asrg] download approaches

2011-01-07 17:00:19

On Jan 7, 2011, at 2:08 PM, John Levine wrote:

I thought IXFR was abandoned because it was too expensive.  Or
perhaps that support was pretty poor.  AXFR is obviously out of the
question.

Does anyone know if any DNS software other than BIND supports IXFR?

I'm not even convinced that bind really *supports* it. :)

IXFR isn't useful unless you're maintaining your entire zone
in a transactional (database or dynamic update driven) way,
and that's not that common a case for a typical zone. (It is
a common case for a DNSBL, or could be quite easily.)

Supported or not, it's not very clever, just ships a list of NS
records to delete and a list to add, relative to the version of the
zone that the client says it has.  The server has to have (or be
prepared to generate) the diffs from all the stale versions of the
zone that a client might ask for.

Yes. For any decently designed server that's going to be a *lot*
cheaper to do than rsync, especially for zones that change a lot.

There are some obvious tweaks you can do to make those deltas
more cacheable, but they'll be pretty easy to generate (and cache) on the fly, 
if
you're using a database backend, anyway (blacklist operators
using flat files and scripts to process their data, rather than something
transactional, are likely to be out of luck).

If we know that a file is a list of IP addresses or a list of prefixes
and lengths, we can presumably design an efficient representation of
the differences.  I'm more interested in how we'd distribute it.  For
example, the oft-cited ClamAV distributes via a very large network of
HTTP mirrors.  That's not awful, but I wouldn't want to have to come
up with such a network for every BL in the world.

There are a bunch of ways to do that, but there's one distinction
between them - that's access control. Distributing the list widely
and cheaply without access control is one problem. Distributing it
with access control, along a controlled distribution path is another.

I've not thought about the second case much, but there are some
ways to layer the first case on top of a (cacheable, CDN-able)
http distribution layer fairly efficiently, and that's a content distribution
platform that "everybody" already supports cheaply.

It seems to me that there's always going to be a point below which
it's better to query for individual answers, and above which it's
better to maintain a mirror, so whatever we do we should be prepared
for both.  I also think that it wouldn't be hard to come up with
something that caches a lot better than current BLs,

Yes, it would. I'm less convinced that it's easy to come up with
something that's both significantly better (in terms of query load and
implementation complexity tradeoffs) and also layered on top of a
simple key-value store such as DNS. Well worth a try, though.

which would move
the query/mirror line up considerably.


Yup. OTOH, a cheap, efficient way of running a local mirror would
move it in the other direction.

Cheers,
  Steve



_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg

<Prev in Thread] Current Thread [Next in Thread>