procmail
[Top] [All Lists]

Re: Trouble with procmail v3.13.1: dot file locking

1999-05-18 08:30:58
1999-05-18-12:37:52 Gjermund Sørseth:
Consider how procmail dot-locks a mailbox - first it creates a file
with a unique name, something like /var/mail/_QVETS. Then it tries to
hardlink /var/mail/user.lock to this file. If the user.lock file already
exists (because the user is already receiving some mail), then procmail
unlinks the temporary file, sleeps for 8 seconds and tries again.

Oops --- there's a big fwack of the problem; why unlink the tmp file? This
problem would disappear with two changes, simple to describe (though I don't
know how they will be to implement). First one is, don't unlink that tmp file;
the lock attempt loop should have just two syscalls, link(2) and sleep(3).

And second, that loop, like any retry loop, should be using exponential
backoff. Personally I don't like **2 backoff; fibo is quick enough, and even
cheaper to compute.

This pair of fixes will basically take care of procmail's part of the problem,
though making sure you acquire a kernel lock (if kernel locks are available)
before attempting to create the dotlock file would help with the remaining
problem.

And the remaining problem is, if you have thousands of procmails stacking up
like airplains outside Newark, the directory will grow large with dotlock
files, and large directories introduce hideous delays under many OSes. Aside
from trying not to create the dotlock file until you've gotten a kernel lock
(if available) the only other fix has to lie outside of procmail, namely fix
the system to use a better-scaling filesystem, one whose performance doesn't
degrade so viciously as the number of directory entries grows large.
Reiserfs[1] claims to be one such, though I haven't tried it out. NetApps'
WAFL is another, and SGI has a third, and that's all the alternatives I've
heard about.

One more thought about leaving that dotlock file around while waiting to
acquire; it'd be a kindness to install a signal catcher for some common
possibilities like INTR, HUP, and TERM, to clean up the scratch file before
exiting.

-Bennett

[1] <URL:http://idiom.com/~beverly/reiserfs.html>