From: Carl S. Gutekunst [mailto:csg(_at_)alameth(_dot_)org]
Sent: Wednesday, October 05, 2011 8:55 PM
To: Keith Moore
Cc: Storz, Michael; Valdis(_dot_)Kletnieks(_at_)vt(_dot_)edu; SMTP Interest
Subject: Re: The anti-abuse rDNS check that FTP gave up
Keith Moore wrote:
Just imagine how many wrongly rejected emails aren't reported.
Stupid spam filtering mechanisms are a DoS attack on email.
The problem is nearly all of our anti-spam measures are empirical. We
all know a lot of people who swear by this, that, or the other check,
even if the only supporting evidence is that using that particular
mechanism cut down the number of complaints. And what works on one
stream fails miserably on another.
FWIW, I last looked at this problem about three years ago. I
specifically wanted to know if some form of RDNS checks might be useful
in cutting down the load on the content-based spam filters. (I was also
checking effect and utilization of SPF and TLS.) Note that the purpose
here was not to improve the catch rate. This was on a stream that was
already filtered by a short-lived (60 minute) IP reputation filter, so
that would reduce the message count from some known spam sources; and
most of the recipients were business users.
As I recall, my sample size was something around 10 million E-mails
a single MX server in a load balanced cluster over a 24-hour period.
There was a weak correlation between spam and a simple existence check
for the PTR record. There was no correlation at all for a stronger
check, e.g., A record matches; a message that failed the strong check
was as likely to be judged ham as spam by the content filter.
this is really interesting, because we got a totally different result when
analyzed our data before we introduced the FCrDNS check. To verify our data, I
took all rejected IP addresses which had a PTR record but did not have a
matching A record from the log of the last 12 hours from one of our servers and
run it against the Spamhaus ZEN DNSBL. The result is
- 420 different IP addresses not on ZEN
- 61722 different IP addresses are on ZEN = 99,3 % of all rejected IP addresses
The distribution to the different parts of ZEN is:
I would say, that means there is a strong correlation between spam and at least
the second part of the FCrDNS check.