[Asrg] Tchoi Reply on draft-church-dnsbl-harmful-01.txt

Here's my reply to Church's paper. :)

**************************

Church’s paper entitled "DNS Blacklists Considered Harmful" discusses 
some legitimate concerns with using DNS-blacklists. One such concern 
with DNS-blacklists is the lack of control that an e-mail administrator 
has over such lists. In particular, the author mentions that "it may 
take a significant amount of time for the server operator's request for 
removal from the blacklist to be answered.". With e-mail being a 
business-critical means for communicating, it's essential that the 
e-mail administrator have the ability to make immediate changes to their 
filters when needed. Since DNS-blacklists are operated and maintained by 
third party sources, and not the administrator him/herself, the author's 
concern does seem to be legitimate. In such a case, a question becomes, 
what can the administrator do when a third party blacklist is blocking 
legitimate mail that needs to get through right away? To answer such a 
question, one should note that it's possible for a spam filter using a 
DNS-blacklist to not block messages just because a sender’s source IP 
address is listed in said list. For example, by using the spamassassin 
tool, a score is associated to each DNS-blacklist rule and by lowering 
said score, one could limit the disruption caused by false positives 
from that list. Alternatively, one could simply setup their own 
'whitelist' of IP addresses that they can use to ensure that mail from 
specific sources will always get through.
Another concern that the author raised in his paper was with regards to 
the actual practices of the operators of such blacklists. Specifically, 
the author argues that "a far more serious problem is that of arbitrary 
policies instituted and decisions made by blacklist operators.". In 
response to the authors’s concern, it should be noted that each list has 
their own policies and as a result, some lists may have more 
questionable policies than others. So before adding any DNS-blacklists 
to their anti-spam solution, it is highly recommended that e-mail 
administrators verify the reliability and integrity of the policies of 
the lists they are interested in and select the one(s) that works best 
for them. That being said, it should be noted that there are in fact 
some high quality lists that are being widely used today by corporations 
and Internet service providers as part of their filtering solution.
A final concern was raised after the author conducted an analysis of the 
mail received and filtered on his mail server. In this analysis, the 
author compared the effectiveness of a content-based filter against five 
different blacklists on a corpus of 1,374 messages. When presenting the 
results, the author noted that "only one of the blacklists (SBL-XBL) 
managed to correctly flag more than 50% of incoming spam and all 
blacklists rejected some of the incoming nonspam messages.". In 
comparison, the content-based filter "correctly filtered 97% of the 
spam, without any false positives.". Although the test results clearly 
showed that the content-filters performed better than the 
DNS-blacklists, it should be noted that sample size was quite small and 
that there was no mention of the number of users nor the variety of spam 
that was used in the experiment. With such a small sample size, it’s 
unlikely that the experiment was tested with the many different variants 
of spams (i.e. scam spam, junk mail, non-commercial, phishes, viruses 
etc.) that larger entities would likely see. For example, one such 
entity is Nortel which has over 65 000 distinct user accounts and 
receives an average of 600k e-mails per day. Nortel uses a hybrid of 
filtering mechanisms comprising content rules and the CBL DNS-blacklist, 
which blocks approximately 300k e-mails per day and includes all of the 
variants described above. It should be noted that the blacklist alone 
accounts for approximately 80% of all of their blocked e-mails with a 
false positive rate of 0.01%. With other large entities reporting 
similar DNS-blacklist effectiveness results, it’s not surprising that 
many large-scale environments are currently using blacklists as part of 
their own anti-spam solution.
Towards the end of the paper, the author suggests that technological 
advances will likely reduce the need for blocking mail at the server 
level. Such advances would include mail client spam filtering software, 
which can learn from previous spam. The author goes on to say that such 
software is "far more effective than simple server-based blacklists, 
particularly since they can adjust for the types of spam received by 
individual users.". In response, it should be mentioned that in order to 
effectively use such a filter, the end user is required to train the 
filter so that it can distinguish between both good and bad mail. With 
the limited amount of spare time in people's lives, it wouldn't be 
surprising if most users would balk at the idea of having to train their 
mail filter software to detect spam and non-spam. Instead, they would 
probably prefer that spam be blocked at the gateway instead of being 
filtered at the client. Furthermore, it should be noted that the author 
acknowledges in the problem section of his paper that approximately 70% 
of all e-mail is spam. With such an incredibly high volume of spam, does 
it really makes sense to waste processing power of the user's desktop so 
that its anti-virus scanner to scan extra message for viruses and does 
it really make sense to waste extra bandwidth needed to deliver all mail 
to the end user. Since e-mail virus infections are costly to any 
corporate entity, most administrators would probably agree that it's 
highly desirable to have the gateway filters block bad messages before 
they get to the end user.
In conclusion, Church’s paper brings up some legitimate concerns with 
the use of DNS-blacklists. Each of these concerns were analyzed and 
addressed in this report to show that DNS-blacklists can in fact be 
useful when the administrator selects the right one for their particular 
environment. In other words, when an e-mail administrator verifies the 
reliability and integrity of any list they wish to use and selects the 
one that works best for them, DNS-blacklists can be useful.


_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg