On 4/4/06, Peter J. Holzer <hjp-asrg(_at_)hjp(_dot_)at> wrote:
So if I have records from the greylisting MTA, I can in theory look at
every delivery attempt and classify it as:
1) Delivery was accepted and message was not spam (TN)
2) Delivery was accepted and message was spam (FN)
3) Delivery was temporarily rejected, but a later delivery of the same
message was accepted. This puts it into one of the two categories
above, but adds extra info (number of delivery attempts, delay from
first delivery attempt to acceptance).
4) Delivery was temporily rejected and no later delivery attempt was
successful. Message was spam (TP).
5) Delivery was temporily rejected and no later delivery attempt was
successful. Message was not spam (FP).
In practice this is difficult. Normally, greylisting is done at the RCPT
command, so there isn't much information available (EHLO, MAIL, RCPT,
IP-address of client, maybe size of the message). So it is even
difficult to determine whether two delivery attempts are for the same
message, and even more difficult to determine whether the message was
spam. Still, for small samples, where a human can go over the records
and apply real-life knowledge, it should be possible to get a (somewhat
fuzzy) categorization which is better than that determined by
greylisting itself.
I have seen this subject has been beaten to death numerous times. I
believe that the standard greylisting plugins (for instance for qpsmtpd)
have a list of the major services with broken MTAs that don't retry, just
like the standard SPF plugins have a list of major forwarding sites
that don't do SRS.
In practice the FPs are easy enough to find because they generate
complaints. To do QA on a new greylisting setup, I agree, would be
most tricky as the FPs are lost. There's no way to capture a message
into a "would have been greylisted" folder without forking the whole
damn universe. Which is impossible given the current laws of physics.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg