procmail
[Top] [All Lists]

Re: lockfile on INCLUDERC to throttle?

1999-12-11 16:36:24
I had suggested to Sean,

You could assign LOCKFILE at the beginning of the INCLUDERC file and unset it
at the end of the INCLUDERC file instead of doing those things in your
.procmailrc.  That would be one way.  It is then possible that several
concurrent procmails might have the INCLUDERC file open for reading, but only
one could be actually processing its recipes at any given time.

He asked,

| Could you explain why that would be any different than wrapping the recipe
| with the explicit LOCKFILE directive as I was doing?  Seems to me that once
| the LOCKFILE is encountered, the processing should hold there until the
| lock clears - as long as it encompases the critical section, what would the
| difference be?

The only difference, as I tried to say, is that if you get and release the
regional lockfile inside the INCLUDERC, procmails waiting for the lock will
have the INCLUDERC open for reading, while if you get and release it in
the main rcfile around the INCLUDERC call, they won't.  Thought I made that
clearer; sorry.

Sean had said previously,

|| As a recent 64 message fetchmail process confirmed, this didn't work as
|| expected.

and I asked what the unexpected thing that happened instead was.

| It merrily executed concurrent processes on the separate messages.  i.e.
| didn't work as I'd expected the explicit LOCKFILE to work.

That's why I was thinking that you could have VERBOSE on just around the
LOCKFILE assignments, and it might tell us something.

| Chaning the TEMP variable to point to a user-local tmp directory, with 755
| permissions changes nothing.

Hmm.

| Your suggestion of:
| 
| >   LOCKFILE=$TEMP/spamsrc$LOCKEXT # or maybe under $HOME somewhere
| >    INCLUDERC=$PMDIR/spam/spam.rc
| >    LOCKFILE
| 
| works as desired.

Hmm.  So the location of the regional lockfile didn't matter after all, but
for some reason putting it inside the braces rather than outside, encompas-
sing the fgrep test, did the trick?  Odd.

| I was wrapping outside the rule itself on the basis that it didn't do much
| good to have a score of messages all running the immediatley preceeding
| greenlist check if they couldn't proceed to do the spam portion (although,
| your way means that those messages which pass can continue to be processed,
| even if another message is being thoroughly spamchecked at the time).  Very
| few items are listed in the greenlists though.

My reason for the suggestion was exactly that: so that greenlisted mail would
not wait for the lock, which it didn't need.  There's no particular diffe-
rence, for mail that needs the lock, whether it waits before or after fgrep-
ping the greenlist, so it seemed to be a gain for some mail with no loss for
other mail.  Either way, mail that needs spamchecking has to wait for the
lock at some point.  In fact, it's a gain for all mail, as mail that needs
spamchecking will have fewer competitors for the lock, because greenlisted
mail won't be playing dog-in-the-manger with it.

| If I have a sequence of events, and I want to lockfile the sequence, how
| would I accomplish that?  Say, matching on a header, then filtering it,
| creating an interrim file, and then doing something with that.  Seems you
| couldn't rely on bracing, but instead would have to make use of a mess of
| sequential recipes using the 'A' flag.

If I understand the question, it's the opposite: you need braces and a
regional lockfile within the braces.  If you string recipes together with `A'
flags and request a local lockfile onto each one, then (1) procmail has to
keep releasing and reacquiring the lock and (2) two concurrent procmails can
get to trading the lock back and forth and thus alternating steps instead of
going one at a time.

| Thank you for the assist.

You're welcome.  Good luck.

<Prev in Thread] Current Thread [Next in Thread>