ietf-asrg
[Top] [All Lists]

Re: [Asrg] Some statistics on SPF and spam

2013-02-12 13:56:00
I didn't see the discussion where you promised to produce this, but I think
the problem is how much non-spam also fails SPF.  

From ruleqa.spamassassin.org/?daterev=20130211-r1444680-n&rule=%2Fspf :

  MSECS    SPAM%     HAM%     S/O    RANK   SCORE  NAME   WHO/AGE
      0   0.0236   0.9635   0.024    0.15    0.00  SPF_FAIL  
      0   0.0383   0.3059   0.111    0.27    0.00  SPF_SOFTFAIL  

Way more non-spam is failing than spam.

Catching spam is easy.  Doing so without excessive false positives is
what's hard.

On 02/12, Martijn Grooten wrote:
I had promised to produce some stats on SPF and spam.

Over the Christmas holidays, I sent over 60k spam messages through 21 spam 
filters in the spam-filter test I run regularly. I checked the SPF status of 
the messages and measured how many filters failed to block each message.

Here are the results:
SPF fail: 3171 emails, on average missed by 0.24 filters (out of 21) with a 
standard deviation of 0.04.
SPF pass: 8106 emails, avg 0.93, stddev 0.23
SPF softfail: 8672 emails, avg 0.45, stddev 0.09
SPF neutral: 13466 emails, avg 0.34, stddev 0.04
SPF none: 26938 emails, avg 0.43, stddev 0.06

A neater table and a graph can be found here: 
http://www.virusbtn.com/news/2013/02_04.xml

Now correlation doesn't imply causation and there are good reasons why the 
relationship here may not causal, but let's for a moment we assume it is.

This means that if you're a spammer, failing SPF isn't a good idea, while 
making sure your emails pass SPF means you're more likely to see your 
messages delivered, but you by no means get a free ride to users' inboxes.

If you find a 'clever' way to avoid failing SPF by using a domain with no SPF 
record, there is only a small improvement in your delivery rates.

Martijn.


________________________________

Virus Bulletin Ltd, The Pentagon, Abingdon, OX14 3YP, England.
Company Reg No: 2388295. VAT Reg No: GB 532 5598 33.
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg


-- 
"Begin at the beginning and go on till you come to the end; then stop."
- Lewis Carrol, Alice in Wonderland
http://www.ChaosReigns.com
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg

<Prev in Thread] Current Thread [Next in Thread>