Yakov Shafranovich wrote:
Chris, can you provide some information on DNSBLs and how your system
evaluates their effectiveness?
Taking the second part first:
1) Our false positive process involves having the sender contact a
specific address, and that will contain a "block ID" from the message
rejection. Using the block identifier, we can retrieve the message, and
identify _why_ it was blocked (may not be DNSBL). Then, if it's a
DNSBL, we can locally whitelist the IP.
2) Our metrics scan the logs and identify block/unblock counts for all
messages by IP. During metrics processing, we generate tables of how
much email a given blacklist blocked, _plus_ how many email messages
_would_ have been blocked by that blacklist, but weren't because they
were whitelisted.
Thus, for each blacklist, we generate:
a) how many emails were blocked, and the percentage of total inbound it
represents.
b) how many emails had to be whitelisted for that blacklist, and what
percentage of the emails blocked by that blacklist it represents.
Note that the latter (b) isn't _really_ a "lost email" metric (to us).
Those messages weren't lost, they got through because we whitelist. But
it's a good FP metric.
Basically coming up with a "blocking percentage" and a "whitelisted"
percentage for each blacklist.
Here's our overall effectiveness metrics - last two week period -
percentages are of _total_ email to our inbound mail servers.
This is meant merely as a demonstration of what significant site
_should_ be doing with metrics from their own filtering, not an
exhaustive list of all DNSBLs, or what others see.
Due to the variety of DNSBLs, it's very difficult to generate overall
metrics for all. Without zone transfering the DNSBL for example, it's
too slow to measure it with reasonable sample sizes.
The counts are _all_ total recipients affected. So, a single email with
5 recipients counts as 5.
Rule count % FPs % of blocked
CBL 352153 16.86 7 0.0020
MONKEYPROXY 261436 12.52 83 0.0317
CONTENT 260391 12.47 N/A
BOPM 167277 8.01 0 0
OSsocks 152168 7.29 0 0
SPEWS (not used) 94285 4.51 N/A
NTthresh 54233 2.60 27 0.0498
SBL 47614 2.28 156 0.3276
NTmanual 45366 2.17 2955 6.5137
NTauto 25094 1.20 12 0.0478
OBproxies 21068 1.01 18 0.0854
PDL 16547 0.79 N/A
NTnordns 5814 0.28 0
Flonetwork 229 0.01 0
[The below numbers are whitelisted recipients per associated BL, as a
percentage of the total inbound, not of blocks per-se. "OK" is _total_
whitelisting entries, whether or not the whitelisting is still necessary]:
OK 36770 1.76
OK SBL 156 0.01
OK OBproxies 18 0.00
OK NTthresh 27 0.00
OK NTmanual 2955 0.14
OK CBL 7 0.00
OK NTauto 12 0.00
OK MONKEYPROXY 83 0.00
TOTAL 2088400 100.00
TOTAL BLOCK 571850 27.38
Ie: out of 2088400 attempted deliveries, 352153 (16.86%) were blocked by
the CBL. Only 7 messages (0.0020% of 352153) would have been blocked if
we weren't whitelisting.
Some notes:
"CONTENT" is the union of all of our non-DNSBL based. Our system
doesn't let us compute FPs on that directly.
BOPM, Monkeys, OSSocks, OBproxies are third party open proxy/socks
blacklists. The first three are public.
CBL is more-or-less an open proxy/socks blacklist (see cbl.abuseat.org)
SBL is Spamhaus.
NTmanual is local manual black/whitelist (the high FP rate is perfectly
okay because those are whitelisted, and it's under our control)
NTnordns is "spam over threshold, no rDNS" (AOL does this now)
NTthresh is "spam over threshold, listed in ORDB or SpamCop"
NTauto is local open relay/proxy detector.
Our system doesn't let us compute FPs on BLs we don't use, so we don't
have the number for SPEWs.
PDL (DHCP pools) does block occasional legitimate email, but, by policy
we don't want email from DHCP pools. So we don't whitelist it - we get
the sender to either use their proper mail servers, or get PDL to update
the entry if it's erroneous. Whitelisting would be silly if the entry
was accurate, because the IP can essentially be random.
Flonetwork: zone listing doubleclick/dartmail/flonetwork. No, we
_really_ don't want email from them, period.
If someone wants subjective commentary on the public DNSBLs (ie:
proposed BCP compliance), let me know.
Some information in this area would be useful as well.
there's been a few of those posted.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg