On Fri, Oct 26, 2012 at 2:34 AM, Hal Murray
<hmurray(_at_)megapathdsl(_dot_)net> wrote:
I'm obviously biased since I run dnswl.org, but an IPv6-based whitelist may
work better than an IPv6-based blacklist. Enumerating the goodness is
generally easier than enumerating the badness.
What fraction of email comes from hosts you have listed? How hard would it
be to scale your list up to cover the whole world?
We do store IPv6 addresses, but we don't publish them yet (since there
is no standard yet - the result of the discussion here may lead to the
emergence of a de-facto standard, or at least a first trial). So I can
not really tell what fraction we have listed in IPv6 world.
I can tell about our estimate on what we cover in terms of IPv4 based
on the dnswl.org stats. These stats are based not on SMTP trafffic,
but on the DNS traffic which we log (sample) on some of the public
mirrors. Larger senders with better cache utilization are likely to be
somewhat underrepresented in our data. We then do not use the absolute
numbers, but logarithmic magnitudes:
Magnitude Percent
10.0 100%
9.0 10%
8.0 1%
7.0 0.1%
6.0 0.01%
5.0 0.001%
4.0 0.0001%
3.0 0.00001%
2.0 0.000001%
1.0 0.0000001%
Extract from our magnitudes report (I'll happily share more data on request):
1 9.63 IPs where we have no record
2 9.24 IPs which are in our DB*, not published - contains a lot of
"bad apples" ("DNSWL Id 0")
3 8.86 IPs which are in our DB*, not published - with fewer "bad
apples" ("DNSWL Id 1")
4 8.44 Yahoo
5 8.38 Google
6 8.36 Internally blacklisted (snowshoe ranges etc)
7 8.12 Hotmail
8 8.03 Facebook
9 7.91 Exacttarget
10 7.89 cheetahmail.com
* Through imports from third parties or "learned" through the DNS logs
It's clear that key is the mag 9.63 of where we have no record. This
category contains all the residential/dynamic/botnet IPs; we do not
count the number of different IPs.
DNSWL Id 0 contains about 300k IPs, DNSWL Id 1 contains about 100k
IPs, all the other (published) DNSWL records contain about 200k IPs.
Records in 0 and 1 have quite some fluctuation (eg removed since they
are not present in the import source any more; promoted from "0" to
"1" based on some criteria; demoted from "1" to "0" based on the lack
of the same criteria). Entries are manually promoted from "0" and "1"
to one of the published DNSWL Ids.
Possibly 80k in each of the two "special" records may be considered
"good" (as in: not operated by spammer/bot herders), which would lead
to ~ 360k mostly legitimate SMTP-sending IPs.
Assuming that you don't want to put all your eggs in one basket, how many
white lists would you need and/or how would you decide the order to check
them?
One? Definitely too risky. Two? Not sufficient, in my view. Three?
That may work. Four? Could improve diversity. Five? There is a point
where returns of additional lists become diminishing (eg higher
latency, larger overlaps).
-- Matthias
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg