fetchmail-friends
[Top] [All Lists]

Re: [fetchmail]fetchmail fork?

2004-05-24 12:27:46
Rob Funk <rfunk(_at_)funknet(_dot_)net> writes:

We seem to be in an interesting situation with fetchmail.  Last year, ESR 
got sucked into writing his book and didn't have time for fetchmail for a 
while, and patches piled up. Then there was a spurt of activity, 
eventually resulting in 6.2.5 around mid-October. But more patches piled 
up, and in January we were told that 6.2.6 would come any day, after which 
some longstanding concerns could be addressed. Then ESR disappeared again, 
until a few weeks ago I pulled him into a discussion relating to one of 
those longstanding concerns.  At that time he muttered something about not 
having list admin privileges but never explained what that had to do with 
fetchmail 6.2.6.

We've seen this, but if he can post, what has that got to do with list
admin? If he meant web site admin, well, maybe, but why isn't he asking
for help?

The precedent case is bogofilter. That project was abducted by Adrian
Otto and released again on SourceForge, around 0.7 time in September
2002. This set the ball rolling, with ESR's time allowing him to
contribute one patch in the whole time. David Relson jumped in as a
maintainer, I helped him, and we got a lot of support from other people,
code, ideas, thoughts, testing, documentation, Greg Louis, Gyepi Sam,
Tom Allison, Boris Piwinger, and bogofilter might see its 1.0 release in
later summer. David is a very nice guy to work with, and the model was
that the core developers all had commit privileges, and with a bit of
communication behind the scenes, everything has worked out well so far,
we haven't trampled over each other.

A lesson I learnt is that CVS sucks. It sucks hard. Really hard. Royal
pain. Branching and merging is a major PITN, and so is SourceForge's
speed with respect to CVS, it was OK when bogofilter started in fall
2002, CVS speed is bearable with a bit of patience, but I don't like
it. SVN (Subversion) stepped up as a "compelling replacement", other
alternatives are mcvs as a client-site bore-up kit for CVS, or the
commercial Perforce which is free to use for open source AFAIR (but
IIRC, the license has to be renewed annually). A different, distributed
and disconnected model is followed by GNU Arch and the commercial
BitKeeper which is also free if you agree to log your ChangeLogs in a
public place and so long as the BitKeeper user and his employer do not
work on a competing product (i. e. a BitKeeper user cannot file patches
for GNU Arch) -- if anyone does, the BK Commercial license is
required. I don't know if GNU Arch is mature, and how mcvs will stack up
against SVN.

And now we actually have a decent development discussion going, from which 
ESR is conspicuously absent.  (Could have something to do with the list 
admin problem?) These developers want to improve things in fetchmail, but 
the most recent release needs too many bug-fix patches before it's a 
stable base for the development people want to do.

ESR has also mentioned wanting to totally rewrite fetchmail, probably in 
python. Based on previous upheavals ESR has initiated, I suspect that the 
result of this will make a lot of people unhappy, even if he ultimately 
has the right idea. (And based on ESR's penchant for surprising people 
with mostly-finished projects, it wouldn't entirely surprise me if the 
6.2.6 delay is actually due to this rewrite happening in secret.)

I don't believe in ESR releasing fetchmail 6.2.6 or 7 the next day in
Python. He could have had all the help and just writing the skeleton,
drafting interfaces and have someone else add the meat.

I understand if ESR is too busy for fetchmail these days, or doesn't want 
to look at C code anymore, or just doesn't care about fetchmail at all 
anymore. I'm not too far from these states (I don't even use fetchmail 
anymore!), but I see people who care a lot and seem to have time to think 
about it.  Anyway, I may be the only one still paying attention who was 
using popclient in 1994, so I feel some sort of responsibility here.

I'd also favour ESR's "complete rewrite" approach, and I have to add,
without poplib, imaplib, smtplib, if it must be Python. I don't care
much what language it will be in as long as it does SSL, TLS, SMTP,
IMAP, POP3, a WORKING injection into a command (fetchmail gets this
right only for the original sendmail.org sendmail), has some hooks to
allow users to peek at the headers and then tell the new fetchmail to
kill off the mail rather than download it.

My personal skills are C, C++, Perl, and I know some Python (but don't
know all its libraries well). Perl is inadequate, C++ would allow for a
smooth switch. Provided someone has the necessary ideas to cast the
current stuff into a structure that would lend itself to C++, just for
the sake of the switch, such a switch would be pointless. If it made the
code more maintainable or something, I'd be all for it.

So I propose that fetchmail development change.  There are many options, 
but the most productive might be to follow the sylpheed-claws model, and 
start a high-development branch run by the community, coexisting with the 
more stable branch controlled by ESR. Maybe we could start a 
"fetchmail-claws" or "fetchmail-ng" project at berlios.de or sourceforge.  
(I've heard enough sourceforge horror stories that I'd suggest berlios or 
another smaller-scale clone.)

Well, in May 2002, I filed a registration for my leafnode project at
Berlios, and never heard back. It's now on SourceForge.

SourceForge is annoying for mailing lists with obnoxious advertising
footers that have been breaking MIME for two years, and its CVS speed is
sub par.

Other than that, I'm all for it. If we can organize some Subversion
repository (I can't), I'll jump in in about two or three months time.

Meanwhile, I've collected some key messages from the fetchmail-friends list 
going back about a year, but mostly from after the release of 6.2.5.
  http://funknet.net/fetchmail/recent/
And I've saved off notable patches from that collection:
  http://funknet.net/fetchmail/patches/
Hopefully this will help in putting together a stable release to build on. 
I'd be happy to put the results of such efforts on my site as well.

I may comment on this later.

I have a private fetchmail tree that I use for POP3 retrieval that is
built on 6.2.5 + Sunil Shetye's two patches + my own fixes for anything
that popped up, and for POP3+(Fast)UIDL+SSL, it's been rock solid for
half a year now. BTW, did I say that all "union" must die? It's a lousy
implementation for RTTI...

I have consolidated Sunil's two patches into one that I call
fetchmail-6.2.5-ss1.patch, and my add-on patch (that goes only on top of
Sunil's applied patch) is fetchmail-6.2.5-ss1-ma3.patch. This is half a
year old by now... URL: http://home.pages.de/~mandree/fetchmail/

If that site requests a password, force a cache flush (usually shift +
click reload), I removed the password. No reason to keep it if ESR is
diving. :)

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95