fetchmail-friends
[Top] [All Lists]

Re: [fetchmail]Q: Saving UIDL state

2001-11-08 09:38:20
On Thursday (November 08) at 09:09am (US/Pacific), Eric S. Raymond wrote:

Andrew Large <andrew(_dot_)large(_at_)openwave(_dot_)com>:
I'm doing a pop3 fetch with "uidl" and "keep" (I dual-pop to my
Linux box and Outlook on my laptop.  I have a "delete after 3 days"
rule in Outlook).

I've noticed that ~/.fetchids doesn't get updated until the entire
pop is complete.  Is there any reason why in my particular
situation?  The problem is that I sometimes have a big stack of
messages to pop and also have a relatively flaky ISDN connection.
If I get 95% of the messages downloaded but lose the link, I also
lose track of the fact that I successfully downloaded the 95% of the
messages that I did (so a later run of fetchmail will result in
duplicate downloads).

I can see the rationale for holding off on writing the IDs when you
are going to delete from server, but when using "keep" is there any
particular reason?  Is there any way of changing this behavior?  I
looked in the code, but didn't see any obvious short-cuts for
writing to the .fetchids file in real-time.  Is this is an artifact
of implementation rather than explicit policy?  If so, any chance of
getting a patch?  If a patch is out, any pointers for what I need to
do in my local source tree?

This is an artifact of the implementation.  There is no known patch.
I might accept one if you want to try writing it...but the UIDL code is
hideously fragile.  You'd need to show me that you had tested very 
thoroughly first.


I ended up writing a kludgy workaround that solves the basic issue for
me.  Here's the details in case you have a better idea:

    * Instead of using "procmail" as my mda, I wrote a simple little C
      program that dumps the message into a personal "spool" directory
      with the message ID as the file name.

    * After a fetchmail run (whether completely successful or not), I
      run a different program that sends the files in the spool
      directory through procmail (and nukes them) *and* adds the message
      IDs to my ~/.fetchids if they aren't already there.

This approach seems to work for the 90% of the messages where the
message ID is also the UIDL.  For the other 10%, I haven't been able to
figure out what algorithm produces the UIDL, nor do I have any other way
of correlating message ID and UIDL (I looked through the code and did
not see any obvious translation on fetchmail's side, so I assume this is
a pop server issue and not something that fetchmail computes).

At this point, the only patch I can think of would be something that
helps me do the correlation.  Alternately, something that writes the
UIDLs to a temporary file (that gets removed after a sucessful run when
the ~/.fetchids file is updated).  I'm not sure either patch solves
enough of the problem to warrant general use.  Thoughts?

-- 
Andrew Large                                      Platform Architect
andrew(_dot_)large(_at_)openwave(_dot_)com                      Openwave 
Systems, Inc.



<Prev in Thread] Current Thread [Next in Thread>