fetchmail-friends
[Top] [All Lists]

Re: [fetchmail]I'm thinking about rewriting the locking code

2001-03-12 02:29:32
From: "Eric S. Raymond" <esr(_at_)snark(_dot_)thyrsus(_dot_)com>
<---SNIP--->
SIMPLE WAY:

One is to scrap the client-side locking entirely and rely on the POP and
IMAP servers to properly serialize or lock out concurrent access to
the same mailbox by more than one session.

This would have the advantage of simplifying the code considerably.
All the lockfile stuff would just drop out.  The reason I put in
locking in the first place was to keep users from screwing themselves
with certain old, flaky POP3 servers that had no busy locking at all,
like the old UCSD pop3d implementation.  But that was five years ago;
it's a different world now, and even the worst POP3 implementations
still live aren't that bad any more (IMAP servers always handled this
better).

The big disadvantage of this path is that "fetchmail -q" would stop
working, and running "fetchmail" in foreground with another instance
in background would no longer simply wake up the background instance.

On one level this is an implementation problem -- I use the lockfile
as a rendezvous where foreground instances of fetchmail can find the
PID of the unique background instance.  On another level it's more
fundamental -- without locking, there is not necessarily a unique
background instance at all.  How would we know which one to signal?

Why not use /var/run/fetchmail.pid - something many applications use. If you want to keep it unique per user then /var/run/fetchmail-<username>.pid for anybody other than root.

COMPLEX WAY:

There is a solution to the -q problem, but it's hairy.  Each fetchmail
instance would have to post its PID to a common database; we'd
interpret `fetchmail' as "if there are background instances running,
wake them all up; otherwise background yourself", and `fetchmail -q'
as "terminate all other instances".  We could play cute games with
allowing a subset of running instances to be selected by the
command-line arguments in the same way that servers to be polled are
now.

If we did this, we'd get mailbox locking for free -- fetchmail
instances could register a lock in the database at the start of a
poll, and remove it at the end.

Implementing such a database is complicated by the fact that we can't
assume all relevant instances will be run under the same user ID.
Some might be daemons started by a sysadmin and running in root mode.
This makes using the filesystem for the database problematic -- all
instances would have to run suid to get read-write access to some
shared database is system-land.

There is a way, however -- shared-memory segments (which have the
desirable properties that they automatically vanish on a reboot and
can be made to vanish automatically when the number of processes
referencing one goes to zero). In this design, all fetchmail instances
would post their PIDs, locks, and foreground/background status to a
common shared-memory segment.

The drawback of this approach is (a) complexity, and (b) not all
Unixes support the shared-memory primitives.  So -q would break on
some systems.

WHAT TO DO?

I'm tossing this out for discussion.  The three basic alternatives are:

(a) leave the present locking alone in order to keep -q working everywhere.

(b) scrap client-side locking entirely -- multiple concurrent
    instances would be possible, but -q would stop working.

Not ideal unless you can come up with some other way to cleanly shutdown all instances of fetchmail. Of course, if kill -HUP does that, then that's ok.

(c) do the funky chicken with shared memory -- multiple concurrent
    instances would be possible, and -q would mostly work but might
    break on some platforms.

The definition of "some platforms" would be interesting here. Mind you, I think it might help if you had a list of what platforms people use fetchmail on and whether those platforms support shared memory.

This seems like the "cleanest" solution where it works, but it all depends on how much it breaks. Would there be any way to to (c) on supported platforms and (b) otherwise?

Rob

_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.