"Tom Betz" <tbetz(_at_)panix(_dot_)com> writes:
I posted a few days ago about 8MB core dumps I'm getting, with the
INCLUDERC recipe that causes the dumps. I've played with it some
more, and this is the current version:
Have you done a stack backtrace from the core file? You may need to
recompile with debugging to get any useful info from a debugger, but
that would also be a chance to turn *off* optimization: some compilers
generate incorrect code when optimizing procmail, and if the problem
goes away when compiled sans -O then there's your problem. What plaform
on you running this on?
...
Examining verbose procmail logs of tests generating core dumps, I see
procmail forking processes for `:0 c' recipes, which continue to
execute while succeeding recipes are being launched. I can imagine that
enough of these would cause the symptoms I am seeing.
If excessive forking caused coredumps, then I don't know how the shell
could exist, given that it's primary task is to fork & exec. Core
dumps indicate either incorrect code (possibly by the compiler or in
the libraries), or someone throwing around signals in a rude fashion.
Since it's too consistent to be the latter, it must be the former.
I *strongly* suggest recompiling procmail without -O and with -g,
then if it continues to coredump, get a backtrace a go from there.
(You're using procmail version 3.11pre4, right?)
I need to use the `:0 c' construct because one email might contain
triggers for many recipes, and I don't want the first recipe it
triggers to cause procmail to exit -- but I also don't want procmail
to dump 8MB cores every time I run it!
Procmail doesn't exit until it hits a "delivering" recipe. To quote
the first page of the procmailrc(5) manpage:
There are two kinds of recipes: delivering and non-
delivering recipes. If a delivering recipe is found to
match, procmail considers the mail (you guessed it)
delivered and will cease processing the rcfile after having
successfully executed the action line of the recipe. If a
non-delivering recipe is found to match, processing of the
rcfile will continue after the action line of this recipe
has been executed.
...
You can tell procmail to treat a delivering recipe as if it
were a non-delivering recipe by specifying the `c' flag on
such a recipe. This will make procmail generate a carbon
copy of the mail by delivering it to this recipe, yet con-
tinue processing the rcfile.
If all the delivering recipes inside a nested block have the 'c' flag,
then there are no "delivering recipes" and procmail will keep
processing past the block. About the only time you need the 'c' flag
on a nested block is when who need to filter a message for some
processing, but still keep around the original for latter processing.
Is there a way I may instruct procmail to complete one recipe before
launching the next one, so they run and release memory sequentially?
Since procmail is forking, the memory is being used by different
processes. Oh wait, are you saying that your machine is thrashing from
the work load of several procmails throwing around huge messages? It
still shouldn't be causing processes to coredump unless you're machine
is *totally* out of memory in which you should add more swap and more
real memory. If there is memory and/or swap left, then it shouldn't
coredump unless there's a bug somewhere to be tracked down and
eliminated or worked around.
Now some comments on your recipes:
#
# Handle potentially Junk E-Mail from mailmasher.com
#
# Match on any email to or from mailmasher.com
:0 H
* (^Return-Path:|^Received:|^Message-ID:|^Reply-To:|^From:).*[(@ ](mailmasher\
.com)
{
The 'H' flag is the default, and we can do some 'factoring' on that regexp:
:0
* ^(Return-Path|Received|Message-Id|Reply-To|From):.*[(@ ]mailmasher\.com
{
BTW: do you ever close this nested block? I'm not sure what procmail does
if it hits EOF before a nested block is closed.
# log all matches
:0 c
mm_junk.log
Perhaps you should have a locallockfile on that? How important is the
integrity of this 'log'. If you just want to log things, perhaps you
should just set LOGABSTRACT to "all" and use the normal LOGFILE to keep
track of things.
# Is the complaint feature turned on?
:0 c
* COMPLAIN ?? on
{
# if so, first make a temporary copy of the email
:0 c
| cat > $MAILDIR/spam.tmp
# then form a reply from just the header, with appropriate substitutions; con
catenate an autoresponse
# message on the end of it; add a copy of the original email, with full head
ers; then add a .signature
# and email it back to sender and to a fixed list of Cc: addresses
:0 ch
| ($FORMAIL -rk -i"Subject: HI, MailMasher user! I'll get back to you." \
-i"Errors-To: tbetz(_at_)pobox(_dot_)com" -i"Reply-To:
tbetz(_at_)pobox(_dot_)com" \
-A"X-Loop: tbetz(_at_)panix(_dot_)com" -i"Cc:
tbetz(_at_)pobox(_dot_)com,numnuts(_at_)mailmasher(_dot_)com" -X
""; \
cat $PMDIR/autoresponse.mm $MAILDIR/spam.tmp; \
echo "";\
echo "-- ";\
cat $HOME/.signature \
) | $SENDMAIL -oi -t
}
# Indicate that a match was found -- the value of $FOUND is later
# assigned to $DELIVER
FOUND=on
}
Actually, the setting of FOUND will *never* be noticed by later recipes:
the variable is set in the child process of the fork and is thus never
seen by the parent which processes the rest of the .procmailrc. If you
want to pass a variable out of the block, you *cannot* use the 'c' flag
on the block. In fact, the 'c' flag on the block does nothing useful,
as the block contains no delivering recipes (so procmail would have
continued out of the block anyway), and no filtering recipes (so the
message in procmail's memory would be the same either way).
Next, you need to do some locking in the above to keep simultaneous
messages from overwriting spam.tmp before the complain goes out.
Since this is between multiple recipes you have to the the global
lockfile variable LOCKFILE.
You could also use some more paranoia, but that's because I'm a sysadmin.
# Is the complaint feature turned on? (Use ^^on^^ for paranoia.)
:0
* COMPLAIN ?? ^^on^^
* ! ^X-Loop: tbetz(_at_)panix(_dot_)com
{
# Is the incoming message too big to deal with? If it's larger than
# 1 Meg then something bogus is going on, possibly a Denial Of Service
# attack. Just drop it on the floor. Alternatives to /dev/null'ing
# include using "head" to extract the first N characters (*not* lines)
# but you need a newer version of "head" that has the -c flag for that,
# or you can simply move your message from the top of the body to the
# bottom, as that doesn't require a temporary file to hold the original
# body. If they expect you to read the entire message, why shouldn't
# you do the same of them? You'll need to rewrite this entire recipe
# if you do.
:0
* > 1048576
/dev/null
LOCKFILE = $MAILDIR/spam.tmp.lock
:0 c
|cat >spam.tmp
# There are some occasions where setting FORMAIL to the full path to
# formail makes sense, but usually it's easier and better to just make
# sure PATH contains the correct directory. a) Never hardcode a path
# that can break in the future (just wait till formail is moved to
# /usr/bin); and b) let the computer do the work. But that's just my
# opinion...
# I've also left off both the -k and the -X "" args, as they cancel
# each other. In fact, since you included the 'h' flag on the recipe,
# the -k wouldn't have done anything to begin with.
:0 ch
| (formail -r -i"Subject: HI, MailMasher user! I'll get back to you." \
-i"Cc:
tbetz(_at_)pobox(_dot_)com,numnuts(_at_)mailmasher(_dot_)com" \
-i"Errors-To: tbetz(_at_)pobox(_dot_)com" -i"Reply-To:
tbetz(_at_)pobox(_dot_)com" \
-A"X-Loop: tbetz(_at_)panix(_dot_)com"; \
cat $PMDIR/autoresponse.mm $MAILDIR/spam.tmp; \
echo ""; echo "-- "; \
cat $HOME/.signature \
) | $SENDMAIL -oi -t
# Remove the lockfile
LOCKFILE
### Indicate that a match was found -- the value of $FOUND is later
### assigned to $DELIVER
# What's this variable "$FOUND". The variable is named "FOUND", and
# you don't assign to $DELIVER, you assign to DELIVER. The name is
# not the object, and in this case, the interpolation is not the
# variable! BTW: do you mean the magic variable DELIVERED?
FOUND = on
}
Philip Guenther