To whoever is co-ordinating ASRG's reply, here is my contribution.
Will it be incorporated into an "ASRG response", or do I have to email
it myself?
=======================================================================
It appears that the author of the draft is labouring under some
mis-understandings. Invalid initial assumptions lead to invalid final
conclusions. I will address those invalid assumptions on a point by
point basis, but first I wish to address a few meta-issues.
Meta-Issues
===========
1) The document seems to be a long diatribe, ending with a couple of
suggestions for DNSBL management. A personal diatribe has no place in
an RFC. Suggestions for DNSBL management duplicate the efforts of the
document at...
http://www.shaftek.org/publications/drafts/draft-irtf-asrg-bcp-blacklists-00.txt
and should be submitted as comments in response to that document.
draft-church-dnsbl-harmful-01.txt should be rejected as a stand-alone
document.
2) DNSBLs don't block anything, just as restaurant reviewers don't
pay goons to stand at the entrances of certain restaurants and attack
would-be patrons with basball bats. Just like restaurant reviewers,
DNSBLs publish opinions about certain establishments. System
administrators are free to accept or not accept those opinions.
3) The draft's author compares a DNSBL-only "solution" versus a
content-filter-only "solution", and declares content-filters as the
winner. This is a misleading straw-man attack, which should be deemed
sufficient reason to reject the draft. In the anti-spam community the
derisive term FUSSP ("Final Ultimate Solution to the Spam Problem") is
applied to any attempt to solve the entire spam problem with only one
tool.
No serious proponent of DNSBLs claims that one DNSBL, or even a
combination of DNSBLs, will solve the entire spam problem by itself.
DNSBLs should be viewed as weapons that are part of the arsenal in the
war against spam. Specific DNSBLs address specific portions of the spam
problem. There is no logical reason to treat content-filters versus
DNSBLs as a strict XOR (exclusive or) proposition. They both have a
place in the war against spam, along with other weapons.
4) The draft-author advocates dropping standard POPmail in favour of
a webmail interface. The whole point of the anti-spam battle is to save
traditional email. Advocating abandonment of traditional email is not a
solution. And what happens when spammers start successfully attacking
webmail? What does the draft-author propose we do then? I say that we
draw a line in the sand, and defend it.
Point-by-point problems
=======================
As spam continues to grow throughout the Internet, various
countermeasures have been developed. Among these is the "DNS
blacklist", a DNS server configured to return a "good" or "bad"
response to a query on a given IP address; mail servers can be
configured to automatically query such a server and reject messages
which are flagged "bad".
DNSBLs should be referred to as "DNS Based Lists", rather than
"DNS blacklists". Consider zz.countries.nerd.dk as an example. It
attempts to list, and return an answer for, every valid IPV4 routable
address. This does not imply that every IP address is "blacklisted".
zz.countries.nerd.dk encodes the ISO numeric code of the country in
which the IPV4 address is believed to be located. The admin of the MTA
can configure the MTA to react differently based on the response
recieved from zz.countries.nerd.dk. A more correct definition of a
DNSBL (DNS Based List) server is...
- a DNS server configured to return certain information in response
to a query on a given IP address. This response may be any type
of response that a DNS server is normally capable of. The MTA may
react to the response in any manner as programmed by the system
administrator.
While it cannot be denied that DNS blacklists are effective to an
extent in blocking spam, the reality is that not only are they no
panacea, but they cause significant collateral damage in the form of
false positives: not infrequently, users find themselves unable to
send mail because their mail server has been listed on one of these
blacklists.
There is no denying that this sometimes happens. Unfortunately, the
choice is not strictly black-and-white. It comes down to a choice
between the lesser of two evils. As far as the users of DNSBLs are
concerned, slightly reduced email connectivity is not as bad as a
useless inbox overflowing with spam. It's sad that this trade-off has
to be made, but that is the real world.
One reason for these false positives is simply the rate of change
of the Internet environment. An IP address used for spamming and
blacklisted yesterday might today be used by a completely innocuous
mail server,
[...deletia...]
The second, and far more serious, problem is that of arbitrary
policies instituted and decisions made by blacklist operators, such
as the policy mentioned above of listing an entire netblock when a
single spam source is found,
It is an unfortunate fact that some ISPs refuse to kick off spamming
customers. Instead, the ISP moves the spammer's block around within the
ISP's address range, in an attempt to bypass DNSBLs. This effectively
makes the ISP's static IP addresses equivalant to dynamic IP addresses.
Not too surprisingly, DNSBLs treat those ISPs' entire net blocks as
dynamic IP addresses.
or the addition of IP addresses to a dialup blacklist because they
subjectively "look like" dynamic addresses.
It is not humanly possible for a list operator to manually analyze
and check millions of IP addresses. Therefore, they have to use an
automated procedure. A regular expression like "[0-9]+-[0-9]+-[0-9]+"
or "[0-9]+\.[0-9]+\.[0-9]+", applied against the rDNS of the IP address
is a very good filter. If an organization, or an ISP, can't be bothered
to set up a non-generic rDNS, much of the blame rests on their
shoulders. This is one area where I agree with the draft author, that
ISPs can help the situation by making their static-IP-address rDNS
easily distinguishable from generic dynamic IP addresses.
encountered other blacklists that block entire countries
DNSBLs don't block anything, they provide information to the querying
machine. Period, end of story. As for blocking entire countries, if I
don't have any contacts in a country, don't understand the language, and
my text-console isn't even set up to display their character set, what's
the point of accepting spam from them?
An analysis of mail received by the author's mail server demonstrates
the limited effectiveness of, and collateral damage caused by, these
blacklists. The analysis was performed on 1,374 messages (including
message send attempts, defined as SMTP connections to the server in
which the SMTP client began a transaction with a MAIL command but did
not complete the transaction before disconnecting) received over a
period of approximately two weeks.
There's so much wrong with the above, that I hardly know where to
start...
1374 in two weeks is an extremely small sample. My rules have
blocked more delivery attempts in two *DAYS*, and I'm considered low
volume. Also, no single tool, including DNSBLs, should be considered as
the only tool to use. That is the FUSSP mentality (see above). This
author uses whitelists and blocks on forged HELOs, lack-of-rDNS,
dynamic-looking-rDNS, manually selected CIDRs, and various other rules.
DNSbls come near the end of the blocking chain, and mop up the
approximately 2% of spam that gets past the other rules.
Whitelists are an absolute necessity. They allow draconian blocking
rules with virtually no false positives. Even content-based filters
require whitelists. This author subscribes to lists that discuss spam,
and include samples thereof. No content-based filter that effectively
blocks on spamsigns will allow those emails through without some form of
whitelisting.
As can be seen from the chart, only one of the blacklists (BL1)
managed to correctly flag more than 50% (57%) of incoming spam,
This author rejects the concept of FUSSP. Any tool that helps block
spam with negligible false-positives is a welcome addition to my
spam-blocking toolbox. I'd rather have 10 tools that block a different
10% of spam each instead of 1 tool that blocks only 97%.
The custom filter actually used on the server, on the other hand,
correctly filtered 97% of the spam, without any false positives.
(The filter has generated false positives on rare occasion, but none
were seen during the test period; the long-term average false
positive rate is estimated to be less than 0.1%.) The filter
consists of regular expressions derived from spam seen in the past,
and the author spends an average of about 40 seconds a day
classifying unfiltered spam and updating the filter.
Two questions...
1) how does the filter react to "spamsigns" in spam-discussion
mailing lists?
2) what percentage of the general population understands regexps?
As technology advances, it is also necessary to revisit the original
assumptions that led to the implementation of "hard" mail filters--
that is, mail filters that block mail from reaching downstream
recipients, as opposed to e.g. adding or modifying a header--in the
first place.
The draft author misses an important point. A false-positive 5XX
SMTP-stage reject immediately notifies a legitimate sender that there's
a problem, and they can pick up the phone or use other channels. A
content-filter false-positive results in the email being buried in a
pile of possibly thousands of real spam in a "spam folder". The sender
thinks that the recipient has received and ignored the email. The
recipient thinks that the sender hasn't sent an email. The recipient
will most likely not bother plowing through their "spam folder".
Otherwise, what's the point of a content-filter?
In the mid-to-late 1990s, when spam first became a significant
problem, mail client software had only simple filtering abilities;
most modern software includes sophisticated spam filters which learn
from previous spam and are far more effective than simple
server-based blacklists, particularly since they can adjust for the
types of spam received by individual users. Metered data connections
are also becoming less common as high-bandwidth, always-on links
become available to more users, and the increased bandwidth of such
connections makes the time spent downloading spam negligible in most
cases.
With the exception of advertisment-ridden and painfull web interfaces,
many email inboxes are still around 10 megabytes or so. Bouncing email
due to full inboxes is a problem. Regardless of whether or not email is
flagged as spam, it still consumes space in one's inbox.
--
Walter Dnes <waltdnes(_at_)waltdnes(_dot_)org> In linux /sbin/init is Job #1
My musings on technology and security at http://tech_sec.blog.ca
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg