On 10/19/2011 02:42 PM, Martijn Grooten wrote:
It is hard to reliably determine how much greylisting helps on a
specific system, i.e. what difference it makes compared to the
hypothetical situation greylisting wasn't used. May be an idea
to include some caution about that.
Log delivery attempts that are greylisted. Log greylisted delivery
attempts that subsequently pass. Compare the IPs in question
against whatever DNSBLs are in use to determine if the delivery
attempts would have been blocked anyway (without resorting to
computationally expensive content filtering)....
Might also make sense to log the DNSBL result at the time that a
delivery attempt is greylisted, and then log delivery attempts
that pass greylisting but are rejecting due to DNSBLs, and thereby
determine if greylisting is effectively giving a recently gone-bad
host extra time to get itself listed.
Yes, that makes sense and, if you're running a test (and don't care
too much about computational performance) could be extended to
content scanning too.
Right, and then log whether content filters would have blocked the
message anyway, had the sender eventually retried. Glean how many (if
any) spams were stopped by greylisting that neither DNSBLs nor content
filters would have otherwise caught.
But that assumes you are able to identify messages, which I doubt is
possible to do in a reliable way unless you greylist during/post
DATA. Which -- at least in theory -- does not need to give the same
results as greylisting at an earlier stage.
Not necessarily.. Say one sees 1000 IPs that attempted to send 100,000
times, were greylisted, never passed greylisting, and weren't listed on
DNSBLs that the system normally checks - that gives one possible piece
of data on effectiveness. If one takes it a step further, find all the
counted greylistings where the IP attempted to send to at least one
spamtrap, and only count those incidents but exclude the actual
attempted trap deliveries, one might get a very conservative count of
how many attempted spam deliveries to actual user addresses were
prevented by greylisting that the DNSBLs in use would not have caught.
Since one isnt matching initial greylistings to later successful
retries, identifying the message isnt necessary to get useful stats.
Regarding the second method, simply logging
blacklisted-ip-was-greylisted, and non-blacklisted-ip-was-greylisted,
and then matching the second group of IPs (not messages) against later
DNSBL-based rejections would give a good indicator as to whether
greylisting is helping to mitigate the delay between when a host starts
spamming and when a host is blacklisted.
I guess the only way to really test the effectiveness of (your
version of) greylisting (on your mail stream) is if you have a
reasonably large stream and you split that feed in two random groups
and use greylisting one group only.
Yeah that would work. If thats not feasible, one could also disable
greylisting periodically at random, and see if there is a spike in
uncaught spam.
(I suppose most people don't care as much about measuring
effectiveness as I do, nor should they, but including something about
effectiveness may be a good idea as this is the main reason people
use greylisting.)
Agreed. May also be a good idea to recommend some measuring techniques
in the BCP, since effectiveness might vary based on site-specific factors.
--
Joe Sniderman <joseph(_dot_)sniderman(_at_)thoroquel(_dot_)org>
_______________________________________________
Asrg mailing list
Asrg(_at_)irtf(_dot_)org
http://www.irtf.org/mailman/listinfo/asrg