Greetings:
With much help from the authors as well as the ASRG chair and Dave
Crocker, I have finally completed the revisions to the DNSBL BCP. A
copy is beneath my .sig so that you may make your comments inline if
desired. I tried to include as many comments as possible from the
discussion of the previous draft, but the authors and the editor deemed
it was necessary to decline to use some of the suggestions.
Flame away! I'm wearing my Nomex suit. :-)
Regards,
Nick
--
Nick Nicholas
Knowledge Engineer
Habeas Inc.
650-694-3320
nick(_at_)habeas(_dot_)com
-------------------
Internet Draft Yakov Shafranovich (Editor)
Expiration: August 2007 SolidMatrix Technologies
Network Working Group Nick Nicholas (Editor)
Habeas
Matt Sergeant
MessageLabs
Chris Lewis
Nortel Networks
February 2007
Guidelines for Management of DNS-Based Reputation Systems for Email
draft-irtf-asrg-bcp-blacklists-01.txt
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. Internet-Drafts are
working documents of the Internet Engineering Task Force (IETF),
its areas, and its working groups. Note that other groups may
also distribute working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use
Internet-Drafts as reference material or to cite them other
than as ``work in progress.''
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (C) The Internet Society (2007). All Rights Reserved.
Abstract
The rise of spam and other anti-social behavior on the
Internet has led to the creation of shared blacklists and
whitelists of of IP addresses or domains. This memo discusses
guidelines for management of public DNS blacklists.
The document will seek BCP status. Comments and discussion of this
document should be addressed to the asrg(_at_)ietf(_dot_)org mailing list.
Table of Contents
Abstract
1. Introduction
1.1 DNS-Based Reputation Systems
1.2 Guidance for DNSBL Users
1.3 Terminology
2. Background.
2.1. Transparency
2.1.2. Listing/Delisting Criteria SHOULD Be Easily Available.
2.1.3. An Audit Trail SHOULD Be Maintained.
2.2. Listings and Removals
2.2.1. Listings SHOULD Be Temporary.
2.2.2. A Direct Non-Public Way to Request Removal SHOULD Be
Available.
2.2.3. Removals SHOULD Be Prompt.
2.2.4. Criteria for Listing and Delisting SHOULD Be Similar.
2.2.5. Removals SHOULD Be Possible in Absence of List
Maintainer.
3. Operational Issues
3.1. Inactively Maintained Lists SHOULD Cease Operations.
3.2. Cessation of List Operations MUST Be Done in a Graceful
Fashion.
3.3. Content of DNSBL Zone File SHOULD Be Limited.
3.4. Special and Reserved IP Addresses MAY Be Used.
3.5. Use of Collateral Damage MUST Be Disclosed.
3.6. Special Considerations for Blacklists Listing Insecure
Hosts.
3.6.1. MUST NOT Scan Without Provocation.
3.6.2. Re-scan Periods SHOULD Be Reasonable.
4. Security Considerations
5. Informative References
6. Author(s) Addresses.
7. Acknowledgements.
8. Full Copyright Statement.
1. Introduction.
1.1. DNS-Based Reputation Systems
Due to the rising amount of spam and other forms of network abuse on
the Internet, many community members and companies began to
create and maintain DNS-based reputation systems ("DNSBL") of IP
addresses and domains identified as problem sources of email. These
lists also have been known as blocklists and blacklists. These
lists are used for filtering email. DNSBLs are either public or
private. A public DNSBL makes its data available to any party
seeking information from the list, but a private DNSBL is used
solely by an organization for its own use and the data is not made
available publicly.
The first publicly available DNSBL using the Domain Name System (DNS)
for distributing reputation data about email senders emerged in 1997,
shortly after spam became a problem for network operators and email
administrators. This pioneer DNSBL focused on identifying known
spam sources situated at static IP addresses. Due to the broad
adoption of this DNSBL, it had a devastating impact on static spam
sources. Consequently, abusers found other methods for distributing
their spam such as relaying messages through unsecured email servers
or flawed formmail scripts on web pages. Additional DNSBLs were
developed by others in order to address these changing tactics, and
today more than 700 DNSBLs are in operation.
Existing DNSBLs vary greatly in their policy, usage and and level
of maintenance. These guidelines try to lay out a set of best
practices for running a DNSBL in an open manner with the hope that
they will facilitate better management of DNSBLs, and provide better
information for prospective users in selecting which DNSBLs to use.
These DNSBLs vary widely in integrity, strategy, methodology and
stability. While listing criteria can sometimes be quite
controversial, this document deliberately does not discuss the
rightness or wrongness of any criteria. We assert that DNSBL
operators are free to choose whatever listing criteria they wish,
as long as they are clear to their users what those criteria are.
The DNSBL user has the responsibility to ensure that the listing
criteria and other aspects of a DNSBL meets its needs.
This document is also intended to provide guidance to DNSBL
administrators so that they may be able to identify what features
users would be interested in seeing as part of a high-quality, well-
managed DNSBL, e.g., a clear listing and delisting policy to which
the DNSBL administrator adheres strictly. This document is intended
to be normative rather than prescriptive: it seeks to characterize
the features of a well-managed DNSBL rather than setting out rules
for how DNSBLs should be operated.
1.2. Guidance for DNSBL Users
When choosing to adopt a DNSBL, an administrator should keep the
following questions in mind:
1. What is the intended use of the list?
2. Does the list have a web site?
3. Are the list's policies stated on the web site?
4. Are the policies stated clearly and understandably?
5. Are web pages for removal requirements accessible and
functioning properly?
6. How long has the list been in operation?
7. What are the demographics and quantity of the list's user base?
8. Are comparative evaluations of the list available?
9. What do your peers or members of the Internet community say
about the list.
It is the responsibility of the system administrators who adopt one
or more DNSBLs to evaluate, understand, and make a determination of
which DNSBLs are appropriate for the sites they administer. If a
system or network administrator allows a third party to make
blocking decisions for its network, then the administrator MUST
understand the policies and practices of those third parties
because responsibility for blocking decisions remain ultimately
with the administrator. A DNSBL does not prevent anyone from
receiving email or any other Internet service. A DNSBL *user*
prevents service delivery using the DNSBL as a tool for doing so.
DNSBL administrators are merely expressing an opinion through the
publication of a DNSBL, and it is their absolute right to do so
free of legal encumbrance, and in violation of this BCP. However,
it is through abiding by the guidelines set forth in this BCP that
the administrators of a DNSBL may gain the trust of their users.
These guidelines address public DNSBLs only and do not apply to
privately operated DNSBLs.
1.3. Terminology.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
[RFC 2119].
2. Background.
The Anti Spam Research Group (ASRG) was chartered to address the
spam problem. The ASRG charter includes:
"codification of best current practices in spam management"
This note falls within that category by listing guidelines for
management of public DNSBLs. This document will seek BCP status.
NOTE: This document is a product of the Anti-Spam Research Group
(ASRG) of the IRTF. As per section 3 of [RFC 2014], IRTF
groups do not require consensus to publish documents.
Therefore readers should be aware that this document
does not necessarily represent the consensus of the
entire ASRG.
NOTE: This document is intended to evolve, based on comments from
the Anti-Spam Research Group (ASRG) mailing list. It is
certain that the current draft is incomplete and entirely
possible that it is inaccurate. Hence, comments are eagerly
sought, preferably in the form of suggested text changes,
and preferably on the ASRG mailing list, at <asrg(_at_)ietf(_dot_)org>.
2.1. Transparency.
A DNSBL SHOULD carefully describe the criteria which are the cause
for adding, and the criteria for removing an IP address or domain
name on the list. Such listing and delisting criteria SHOULD be
presented in a clear and readable manner easily accessible to the
public through a mechanism such as the DNSBL's web site. A DNSBL
MUST abide by its stated listing and delisting criteria. Entries
that do not meet the published criteria MUST NOT be added to the
DNSBL.
In other words, be direct and honest and clear about the listing
criteria, and make certain that only entries meeting the published
criteria are added to the list. For example, some DNSBL operators
have been known to include spite listings in the lists they
administer.
There is nothing inherently wrong with this practice so long as it
is clearly disclosed. For example, a DNSBL described as listing open
relays only MUST NOT include IP addresses for any other reason. This
transparency principle does not require DNSBL administrators to
disclose the precise algorithms and data involved in a listing.
Availability of documentation concerning a DNSBL SHOULD NOT be
dependent on the continued operation of DNS for the DNSBL zone file.
In other words, if the DNSBL documentation is located at
http://example.com/dnsbl/, the documentation web site SHOULD remain
available even if the DNSBL zone file is not available. See also
Section 3.2
2.1.2. Listing/Delisting Criteria SHOULD Be Easily Available.
Listing and delisting criteria for DNSBLs SHOULD be easily
available and SHOULD be located in a place clearly marked
in its own section of the web site affiliated with the DNSBL.
DNSBLs often publish their listing criteria along with
additional technical information about using the blacklist. This
additional technical information can confuse end users, so a
separate page or section on its own SHOULD be dedicated to
detailing why a specific entry appears in the DNSBL.
2.1.3. An Audit Trail SHOULD Be Maintained.
A DNSBL SHOULD maintain an audit trail for all listings and SHOULD
make it publicly available in an easy to find location, preferably
on the DNSBL's web site. Please note that making audit trail data
public does not entail revealing all information in the DNSBL
administrator's possession relating to the listing; e.g., a DNSBL
administrator MAY make the audit trail data selectively accessible
in such a way that spam trap addresses are not disclosed.
2.2 Listings and Removals
2.2.1. Listings SHOULD Be Temporary.
With the exception of DNSBLs that that are based on data that does
not change, such as those include the IP addresses associated with
a specific country or geographic region, all listings SHOULD be
temporary so that an entry will time out at some point in the
future. For more aggressive spammers (e.g., those operating their
own ISP) the temporary period MAY be as much as six (6) months.
However, longer periods SHOULD NOT be used since it is possible
that an IP address or domain can change hands, possibly being
allocated to a non-abusive user.
Note that all listings being temporary does not mean that some
listings will not remain after the initial timeout period. If the
DNSBL administrator determines that the conditions for listing
still exists, then the timer for determining timeouts MAY be
renewed.
2.2.2. A Direct Non-Public Way to Request Removal SHOULD Be Available.
Discussions about whether a DNSBL should remove an entry MAY
include activity in a public forum. Methods for processing
removal requests through private, direct exchanges, such as
person-to-person email or a combination of web page requests
and email responses, SHOULD be available. As a minimum, the
DNSBL SHOULD have a web page that has a removal request function
(separate from the page describing listing criteria as per
section 2.1.2). The DNSBL SHOULD also make available an email
address in order to handle issues other than blocking issues.
The DNSBL administrator MUST NOT use the list in question in such
a way that removal requests would be blocked, and, moreover,
SHOULD make unfiltered mailboxes available in order to allow
affected users to submit their requests. In some cases it is
impractical not to filter email to role accounts due to the amount
of spam those mailboxes receive. If filtering should be necessary
in such circumstances, filtering methods with virtually non-
existent false positive rates SHOULD be chosen.
2.2.3. Removals SHOULD Be Prompt.
Requests for removal SHOULD be honored without question.
Although this approach allows people to submit a request and have
any listed IP address removed immediately, it does not prevent
the DNSBL administrator from re-listing the IP address at a
later time (for example: subject to seeing more spam, or
re-checking the IP address security). A re-listing MAY result in
a longer timeout until the listing expires before it is eligible
for removal. Bounded exponential backoff is a good choice for
listing timeout.
Assuming the above is not possible and no listing reasons remain,
removal at anyone's request SHOULD be prompt. Removal requests
SHOULD be acted upon within a period of 24 hours, and SHOULD be
handled within three (3) days.
Most DNSBLs can effectively use a "no questions asked" removal
policy because by their very nature they will redetect or relist
problems almost immediately. They can mitigate more organized
attempts to "game" the system by elementary checking and rate-
limiting procedures, increasing lockout periods, rescans etc.
Furthermore, a few IP addresses more or less do not make a
significant difference in the overall effectiveness of a DNSBL.
Moreover, a "no questions asked" removal policy provides the
huge benefit of a swift reaction to incorrect listings.
As an example, one popular DNSBL implementing a "no questions
asked" removal policy also performs rate-limiting and malicious
removal detection.
Another important consideration supporting a "no questions asked"
self-removal policy is that it forestalls conflicts between DNSBL
administrators and organizations whose IP addresses have been
listed. Such a policy also can be an effective deterrent to legal
problems.
2.2.4 Criteria for Listing and Delisting SHOULD Be Similar
The criteria for being removed from a DNSBL SHOULD bear a reasonable
relationship to the factors which were the cause of the addition to
the
DNSBL. If a listed entity fulfills all published requirements for
removal from a DNSBL, then the DNSBL administrator SHOULD NOT
impose any additional obstacles to remove a given entry from
the DNSBL. There SHOULD NOT be any extra rules for de-listing
other than the ones listed in the published listing criteria.
In addition, it is RECOMMENDED that all listings SHOULD be temporary
as described in section 2.2.1. For temporary listings, it is not
necessary to correct the causes of the listing in order to be removed
from the DNSBL.
2.2.5. Removals SHOULD Be Possible in Absence of List Administrator.
Removals SHOULD be possible in the absence of the list administrator.
If removals cannot be automated (e.g., via robot re-testing) then the
DNSBL SHOULD have multiple administrators so that a removal
request can be processed if the principal list administrator is
on vacation or otherwise unavailable.
3. Operational Issues.
3.1. Inactively Maintained Lists SHOULD Cease Operations
If a DNSBL is no longer actively maintained, it SHOULD cease
operations in accordance with the principles set forth in section
3.2.
3.2. Cessation of List Operations MUST Be Done in a Graceful Fashion.
When a DNSBL ceases operations and is taken out of circulation,
it MUST do so in a graceful manner so that it does not create
excessive DNS queries or list the entire Internet.
The recommended approach is to put the DNSBL in its own second
level domain, and then point the DNS NS records for that second
level domain to 127.255.255.255. The TTL for that record should be
set at the maximum allowed period of one week.
3.3. Content of DNSBL Zone File SHOULD Be Limited.
The content of a DNSBL zone file SHOULD be limited to DNSBL entries
and relevant control information such as the TTL for the entry,
explanatory TXT records, etc. By virtue of using domain names, a
DNSBL is a hierarchy with a root anchored in the global Internet.
The DNSBL "query root" SHOULD be below the registered domain, so
that the DNSBL information is not conflated with domain housekeeping
information (e.g., name server, MX or SPF records). By using this
approach, DNSBL queries would take the form of
"<query>.dnsbl.example.com" rather than "<query>.example.com".
Further, this sub-tree should occupy its own zone. This approach
facilitates clear delineation of function as well as
orderly DNSBL shutdown because the DNSBL nameserver records can be
specified separately from the domain's principal nameservers.
3.4. Special and Reserved IP Addresses MAY Be Used.
The DNSBL MAY list loopback, RFC 1918, LINK-LOCAL class D/E, and
any other permanently reserved or special-use IP addresses. Such
use MUST be disclosed in the documentation related to the DNSBL.
Most DNSBLs follow a convention of entries for IPs in 127.0.0.0/8
to provide online indication of whether the DNSBL is operational.
In particular, most DNSBLs use a positive response for 127.0.0.2
for this. See reference [DNSBL-EMAIL] for additional information.
3.5. Use of Collateral Damage MUST Be Disclosed.
Some DNSBLs have adopted a policy of using "collateral damage" as
an intentional element of the list's operation. When collateral
damage is applied, a DNSBL listing is expanded to include IP
addresses
under the control of a provider even though those IP addresses are
not the source of abusive email. The theory is that customers
impacted by collateral damage will bring pressure upon the provider
to take action against the customer which is the cause of the
original listing.
Collateral damage as a DNSBL policy is highly controversial, and
discussion of the appropriateness of this policy is beyond the scope
of this document. However, if a DNSBL has policies and practices
that include the imposition of collateral damage, such policies
MUST be disclosed.
3.6. Considerations for DNSBLs Listing Insecure Hosts.
Some DNSBLs list IP addresses of hosts that are insecure
in various ways (e.g. open relays, open proxies). The following
recommendations for such DNSBLs may not be relevant to
regular DNSBLs.
3.6.1. MUST NOT scan without provocation.
DNSBLs MUST NOT automatically probe for insecure systems without
provocation. There is little agreement in the community as to whether
or not such activity should be allowed, so this BCP adopts a more
conservative policy.
Therefore, scanning MUST be be targeted, rather than broad-based,
with a given scan motivated by a specific reason to have concern
about the IP address being scanned. Examples of such reasons
include delivery of a message to a spam trap address, receipt of
a user complaint, or periodic testing of an IP address that is
listed already.
3.6.2. Re-scan Periods SHOULD be Reasonable.
If the DNSBL administrator re-scans a host in order to determine
whether the listing SHOULD timeout or not, the re-scan period
SHOULD be reasonable. Automated re-scans SHOULD NOT occur more
often than once every 24 hours.
4. Security Considerations
Like all DNS-based mechanisms, DNSBLs are subject to various
threats outlined in [RFC 3833].
5. Informative References
[RFC 2026] Bradner, S., "The Internet Standards Process
-- Revision 3", BCP9, RFC 2026, October 1996.
[RFC 2014] Weintrib, A., Postel, J,; "IRTF Research Group
Guidelines and Procedures", BCP 8, RFC 2014, October 1996
[RFC 2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997
[RFC 3833] Atkins, D. and Austein, R.; "Threat Analysis of the
Domain Name System", RFC 3833, August 2004
[DNSBL-EMAIL] Levine, John; "DNS Based Blacklists and
Whitelists for E-Mail", draft-irtf-asrg-dnsbl-02, November 2005
6. Author(s) Addresses.
Yakov Shafranovich (Editor)
SolidMatrix Technologies, Inc.
research(_at_)solidmatrix(_dot_)com
www.shaftek.org
Nick Nicholas (Editor)
Habeas, Inc.
nick(_at_)habeas(_dot_)com
Chris Lewis
Nortel Networks
clewis(_at_)nortelnetworks(_dot_)com
Matt Sergeant
MessageLabs, Inc.
msergeant(_at_)messagelabs(_dot_)com
8. Acknowledgements.
We would like to thank John R. Levine, Alan Murphy and Dave
Crocker for their insightful comments.
Funding for the RFC Editor function is currently provided by the
Internet Society.
9. Full Copyright Statement.
Full Copyright Statement
Copyright (C) The Internet Society (2007). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg