-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Laird Breyer wrote:
On Apr 04 2006, John L wrote:
We sure have a lot of raw material. Can I have a volunteer editor or two
to moosh the good stuff into one reasonably coherent document explaining
what we think would need to be changed for it to be good?
I've just read through the threads again and done a summary of all the
main relevant points everyone discussed on the list (see attachment).
Nobody else has stepped up to the plate it seems. How do we go about
publishing this John? Is what Laird (and my subsequent comments) wrote
good enough as is?
If someone with the time wants to rearrange the paragraphs and flesh
them out into a final document, please pipe up and do so. I tried to
relate every point explicitly to the Church draft. Apologies if I
missed someone's contribution.
------------------------------------------------------------------------
Summary of the ASRG discussion on draft-church-dnsbl-harmful-01.txt as
of 04-05-2006.
This is a review of the draft-church-dnsbl-harmful-01.txt document as
discussed
on the ASRG mailing list http://news.gmane.org/gmane.ietf.asrg/
Note: Some of the posts are missing from the archive, such as Dan Oetting's
message-id <84E9C57C-1CE6-4232-90FC-7BE5F9EEC422(_at_)qwest(_dot_)net>. I was
going
to refer to archive urls but ended up just giving Message-IDs.
The church document has the following structure.
1) An informal description of IP DNS blacklists.
2) A criticism of the policies and goals of DNS blacklist operators.
3) A claim of damage to third party mail transport.
4) A claim that botnets cause DNS blacklists to be out of date immediately.
5) An experiment on the accuracy of filtering using DNS blacklists.
6) A suggestion for the improvement of IP recognition
7) A moral argument of responsibility by DNS blacklist operators.
8) A suggestion to deprecate parts of the internet mail system to newer
technologies.
Taken together, the author comes to the conclusion that DNS blacklists should
not be used.
The ASRG discussion centered on several of these points.
Claims 2) and 3) are sensitive issues fraught with political aspects,
and are therefore not appropriate for technical criticism, although
they do represent some valid concerns that are often brought up.
Dan Oetting's thread deals with 4). There is agreement that botnets are
a problem, but there is some research into the problem.
Jim MacLeod <http://article.gmane.org/gmane.ietf.asrg/11516> thread makes
these points:
Regarding 4) IP assignments to particular computers tend to change
slowly enough (on the order of a month) to make IP lists
valuable. [reference?}
Regarding 2), diversity is a strength in anti-spam.
Regarding 5), the experiment is flawed due to its size.
Daniel Feenberg
<Pine(_dot_)GSO(_dot_)4(_dot_)64(_dot_)0603290717090(_dot_)4480(_at_)nber5(_dot_)nber(_dot_)org>
mentions the increased legitimacy of rejecting mail at the SMTP
transaction protocol level rather than silently discarding it for
filtering reasons. Unlike content filtering, DNSBLs can be used at the
transaction time.
Steve Atkins <4D2D1BA3-767A-47FC-BC9F-D54829C2E450(_at_)blighty(_dot_)com>
suggests there are significant general problems with DNSBLs, such as being
misused as reputation systems, technical flaws in their usage,
etc. It may be (Chris Lewis
<442C2405(_dot_)1040601(_at_)americasm01(_dot_)nt(_dot_)com>) that the real
problem is that
people responsible for choosing DNSBLs aren't necessarily exercising
due diligence and knowing what they're getting.
Tony Finch disagrees with Steve Atkins, calling DNSBLs crucial.
Nick Nicholas asked for solid evidence, thereby being relevant to 5).
There is a big thread on testing philosophy of DNSBLs. Everyone agrees
that 5) in Church's paper is flawed. The flaws are as follows:
Flaw: The Church sample size (1,374 msgs) is much too small to be
meaningful. Chris Lewis
<442EE80C(_dot_)5080500(_at_)americasm01(_dot_)nt(_dot_)com>
disagrees with Church's conclusions based on 65,000 users and 600,000
emails per day at Nortel.
Flaw: the comparison of filters in Church's table is
inapplicable. Steve Atkins
<2BC9207C-3E51-4B88-97A8-7445E9A1313B(_at_)blighty(_dot_)com> points out that
pretending that a DNSBL filter was used for comparison purposes, as is
done by Church, is meaningless when it actually wasn't, because filtering
during the SMTP transaction gives feedback (Daniel Feenberg's point)
and therefore changes the spammer's behaviour.
Flaw: Church doesn't explicitly define his terms, such as just what he
considers spam. In turn, this means derived definitions of FP, FN, TP
etc. are not fully mapped out. Laird Breyer and Chris Lewis discuss
the consequences of using different definitions, resulting in
contradictory claims.
Flaw: Church's experiment is applicable at best to one type of email user.
Chris Lewis's experience strongly contradicts the Church conclusion, and is
based on very different business user email patterns.
Flaw: Church's table data suggests rejecting DNSBLs individually on
accuracy grounds. However, this is flawed in that DNSBLs are often
used in conjunction with other filters. The table doesn't address
these other possibilities at all.
Several people point out that DNSBLs are meant to be used as part of a mix of
techniques, ie that the whole is bigger than the sum of its parts.
Peter J. Holzer <20060331191807(_dot_)GA14686(_at_)teal(_dot_)hjp(_dot_)at>
offers some statistics
on the most effective elements of his mix, others chime in on that thread.
Walter Dnes <20060401015927(_dot_)GA27452(_at_)waltdnes(_dot_)org> addresses
2) and 3),
which are out of scope of the technical criticism. He points out that
5) is flawed because it compares a DNSBL-only solution against an
unnamed, unknown custom method, and lists several other flaws
above. This is not realistic since DNSBLs are used in conjuction with
other methods. Walter also points out that 8) is ridiculous.
Thomas Choi <44322933(_dot_)7060103(_at_)nortel(_dot_)com> summarizes various
issues
that appeared during the discussion.
Regarding 2) and 3), it's important for users of DNSBLs to apply diligence
and caution. A single DNSBL verdict should never be used on its own, as
is implied in 5).
Regarding 5), spam filtering statistics always depend on type of email
mix being received. This varies depending upon the type of user
(business or private individual) and the type of social interactions
(frequent mail with regular correspondents or regular short exchanges
with strangers), etc.
Regarding 8), Thomas points out that mail technologies arise out of
many different needs of users, and what is optimal for one person or
entity in terms of usability, time, expertise, processing power etc. is not
necessarily optimal for another.
------------------------------------------------------------------------
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3-nr1 (Windows 2000)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQCVAwUBRE0vbp3FmCyJjHfhAQKUBAQA2yPLPs+GVPyFcK8JbvbbCWCGw8qZFfQh
IEfTTsuxXPGYQWM0lfs7bpYuh/xLae/Iex+1MjYELWlq9G98x8sa2R8CkbrKJctE
Bey0RQCAMEBCUCe+WdZ4svn8DACuiKe7Yw2k3VNH6FXi8ZAeJ/zxmF4ti2pInfQU
eu+ZMxMxd8Y=
=Pw80
-----END PGP SIGNATURE-----
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg