procmail
[Top] [All Lists]

RE: procmail slow on big files?

2001-04-25 16:12:20

Jay writes: "With the amount of rules we have in the /etc/procmailrc file it
gets quite slow when it's processing a 3 megabite file. Is there any
tweaks we can do to speed this up?"
and provides a URL to listing of the mother-of-all procmail recipes:
http://users.exis.net/~jcollin/procmailrc

I think in this case, it'll be faster if you resort to invoking a separare
program, and easier to maintain also.  A pattern something along these
lines:

#
# Remove spammers, but keep friends
#
FROM=`formail -rx To: | sed -e 's/^[ ]*//'`
FGREP=/usr/local/bin/fgrep
:0 H:
* !? echo "$FROM" | $FGREP -i -q -f $HOME/.friends
{
:0 BH:
* ? $FGREP -i -q -f $HOME/.spammers
dumpster
}

This is just a rough example, and I haven't tried testing it to make sure it
produces the intended result. The idea here is that the file $HOME/.friends
lists sender addresses, for messages that will always be allowed to pass,
and $HOME/.spammers consists of a series of one=per-line strings that, if
matched in the header or body will designate the message as junk mail.

If you do use fgrep, I recommend the GNU version 2.0 or higher. It is both
faster, handles more strings, and doesn't appear to have bugs in the pattern
matching logic. Matching very short, commonly occurring strings of course
will generate a lot of "false positives", and unfortunately without a lot
more work, it won't be possible to meaningfully match the content of many
MIME-encoded messages, without a lot more work.

_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail