ietf-smtp
[Top] [All Lists]

RE: The anti-abuse rDNS check that FTP gave up

2011-10-06 07:14:28

From: Carl S. Gutekunst [mailto:csg(_at_)alameth(_dot_)org]
Sent: Wednesday, October 05, 2011 8:55 PM
To: Keith Moore
Cc: Storz, Michael; Valdis(_dot_)Kletnieks(_at_)vt(_dot_)edu; SMTP Interest 
Group
Subject: Re: The anti-abuse rDNS check that FTP gave up

Keith Moore wrote:
Just imagine how many wrongly rejected emails aren't reported.

Stupid spam filtering mechanisms are a DoS attack on email.

The problem is nearly all of our anti-spam measures are empirical. We
all know a lot of people who swear by this, that, or the other check,
even if the only supporting evidence is that using that particular
mechanism cut down the number of complaints. And what works on one
stream fails miserably on another.

FWIW, I last looked at this problem about three years ago. I
specifically wanted to know if some form of RDNS checks might be useful
in cutting down the load on the content-based spam filters. (I was also
checking effect and utilization of SPF and TLS.) Note that the purpose
here was not to improve the catch rate. This was on a stream that was
already filtered by a short-lived (60 minute) IP reputation filter, so
that would reduce the message count from some known spam sources; and
most of the recipients were business users.

As I recall, my sample size was something around 10 million E-mails
from
a single MX server in a load balanced cluster over a 24-hour period.
There was a weak correlation between spam and a simple existence check
for the PTR record. There was no correlation at all for a stronger
check, e.g., A record matches; a message that failed the strong check
was as likely to be judged ham as spam by the content filter.

<csg>

Hi Carl,

this is really interesting, because we got a totally different result when 
analyzed our data before we introduced the FCrDNS check. To verify our data, I 
took all rejected IP addresses which had a PTR record but did not have a 
matching A record from the log of the last 12 hours from one of our servers and 
run it against the Spamhaus ZEN DNSBL. The result is 

- 420 different IP addresses not on ZEN
- 61722 different IP addresses are on ZEN = 99,3 % of all rejected IP addresses

The distribution to the different parts of ZEN is:

     1 XBL-NJABL
     6 SBLCSS
    24 SBL
  4825 PBL-ISP
 25608 PBL-Spamhaus
 31258 XBL-CBL

I would say, that means there is a strong correlation between spam and at least 
the second part of the FCrDNS check.

Michael