procmail
[Top] [All Lists]

Re: URGENT: Looking for Procmail expert

1999-01-29 00:13:40
PSE-L(_at_)mail(_dot_)professional(_dot_)org (Professional Software 
Engineering) writes:
At 20:58 28-01-99 +0000, Bennett Todd wrote:

2) Fix the problem _Completely_ --- configure your MTA (sendmail, qmail,
  Postfix, whatever) to discard messages greater than some reasonable size.
  I recommend 64KB.

You are KIDDING, right?  This turns your mail system into virtually a
text-only environ.  Fine perhaps for a freemail outfit or something, but
this simply will not fly with todays work environments.

While I cannot condone the 64KB limit (I've received text message
larger than that), _some_ limit should be applied by the MTA.  If
nothing else, to make DOS attacks harder (not impossible, just
harder).  As time goes on and disks and messages get larger, this limit
will have to be raised, of course.  Our mailserver runs with a 15MB
limit -- that's big enough for my boss to email the latest netscape to
a faculty member when he's feeling lazy.


There is an additional option:

(4)  rewrite procmail to reduce number of concurrent processes, possibly
via a lock mechanism of some sort -- this would reduce memory consumption
from multiple messages being handled by procmail on the system at the same
time, and I'd imagine also issues with the COPY flag (of course, it also
reduces your ability to use it in complex fashion).

...and could easily lower throughput below the incoming rate, resulting in
a constantly increasing backlog.  Just Say No.


If this is all going on for a single account, you might be able to do as I
have (no changes to procmail itself):

Because my personal procmail recipes are so demanding and complex,
particularily with respect to spawning grep processes to match headers
against massive (2MB+) lists of junkmail domains (brute force, but VERY
effective), I actually envelope my entire procmailrc in a lock --
therefore, my mail filtering takes place ONE MESSAGE AT A TIME (at least
for that mailbox) I can thus get multiple copies of a large message, or
scores of small messages, and I don't get hammered on memory requirements
(well, I still require the memory to process any ONE message, but if you
can't handle ONE, or even TWO messages, you REALLY do need to upgrade memory).

This is simple to accomplish.  Up at the top of my .procmailrc:

# This makes sure we're running only one copy at a time.  This is necessary
# in order to avoid problems arising from the heavy load which aggressive
# spam filtering places on the system - multiple messages eat lots of memory

LOCKFILE=${HOME}/.procmail.global-procmail.lock

As mentioned in another message, this doesn't work: procmail reads in
the message before opening your .procmailrc.  Well, I guess it would
kind of work by cutting down on thrashing, as the OS would be free to
swap out processing waiting on the lock, but they'll still be using
swap.


Philip Guenther

<Prev in Thread] Current Thread [Next in Thread>