On Tue, Apr 27, 2004 at 12:01:04PM -0400, Eric S. Raymond wrote:
Now, there certainly may be valid policy reasons why you might wish to
use the TOP trick. In that case, there should be an explicit flag for
it:
...
Hmmm...possible, though I'm resistent to adding new options.
I'd certainly agree that fetchmail has its fair share already :-)
There are really two orthogonal problems here:
1. Fetchmail's default use of "TOP n 99999999" rather than "RETR n" is
tickling bugs in some (real, current) POP3 servers
2. Some people want to download mail from the server but leave it marked
'unseen' always.
So let's set aside (2), which needs a new option. The simple fix for (1) is
to use RETR instead of TOP when UIDL is being used, rather than LAST, to
manage the session.
This might be as simple as (untried but just to illustrate):
--- pop3.c.prev Thu Apr 29 09:01:47 2004
+++ pop3.c Thu Apr 29 09:08:15 2004
@@ -1100,7 +1100,7 @@
* The line count passed (99999999) is the maximum value CompuServe will
* accept; it's much lower than the natural value 2147483646 (the maximum
* twos-complement signed 32-bit integer minus 1) */
- if (ctl->keep || ctl->fetchall)
+ if (ctl->server.uidl || ctl->keep || ctl->fetchall)
gen_send(sock, "RETR %d", number);
else
gen_send(sock, "TOP %d 99999999", number);
although since fetchmail by default probes LAST and falls back to UIDL, it
would be better to use some flag which says "we're currently using UIDL"
rather than "the user explicitly asked for UIDL"
Maybe that flag is !peek_capable, and maybe we should force peek_capable=0
if we are using UIDL. The logic of how that flag interacts with the rest of
the code is not entirely clear to me. I also don't understand why it is
initialised in two different places to two different values:
[in pop3.c]
/* at start */
peek_capable = !ctl->fetchall;
/* after login */
peek_capable = !(ctl->fetchall || ctl->keep);
Eric, do you think it would make sense to initialize it as
peek_capable = !(ctl->fetchall || ctl->keep);
then set peek_capable=0 if we are using UIDL (i.e. if LAST failed or was not
tried)? Then the RETR test just becomes
- if (ctl->keep || ctl->fetchall)
+ if (!peek_capable)
gen_send(sock, "RETR %d", number);
else
gen_send(sock, "TOP %d 99999999", number);
All this makes sense, except that the unreliable connection issue still
exists in a different form if the user wants messages marked as seen
normally, but not if fetchmail tries and fails to get a message. This is
a corner case that I hope ESR might weigh in on.
That is in fact the case fetchmail's TOP logic was designed to handle
correctly, whether due to line drop or any other reason. (Bear in mind
that when fetchmail was in most active development in '96-'97, polls
over flaky dialup links were morte common than they are now.)
Sure, but in the case of UIDL I don't see that it matters much whether the
message is marked 'seen' or 'unseen' after an aborted download of a message,
since it will be re-fetched on the next attempt anyway.
In any case: we cannot implement the above policy
"if the user wants messages marked as seen normally, but not if
fetchmail tries and fails to get a message"
If you use TOP to download a message, then the message will *always* remain
"unseen", whether or not it was downloaded successfully. Surely you're not
asking fetchmail to retrieve the message twice, once with TOP, and then a
second time with RETR just to mark it as seen??
That's why fetchmail only uses TOP if "keep" and "fetchall" are not set; not
having "keep" means we will delete the message, so we don't care what the
state of its 'seen' flag is after having downloaded it.
I'm CCing ESR on this just in case it helps get his attention.
You have it.
Great. While I have it, could I draw your attention to a couple of other
issues I've tracked down and provided patches for recently:
(1) Broken 'showdots' flag when .fetchmailrc contains two or more 'poll'
entries. It's weird :-)
Explanation and patch (patch-bc) here:
http://lists.ccil.org/pipermail/fetchmail-friends/2004-April/008518.html
(2) Minor ssl issues:
- fetchmail without -v reports "CN mismatch" error (three times) when
connecting to a site which has a good certificate but a different name than
the one used (solution: don't report this error unless -v or sslcertck are
specified, just like all other certificate errors)
- sslcertpath is not honoured unless sslcertck is given, which means that
fetchmail -v is not able to validate certificates (solution: honour the
sslcertpath flag always)
That's patch-bb at the same URL above.
(3) There's a needless sleep(3) after every POP3 login which I am at a loss
to explain; the server has already responded "+OK" to say that it is ready
for more commands. I'm not aware of any other POP3 client which puts in this
delay; I can only imagine it must be a workaround for a bug in some
unspecified POP3 server. Eric, do you have more information? Is there any
reason for this sleep(3) to remain, or at least could it be #ifdef'd out?
It's annoying when you have 'poll' entries for multiple servers.
Thanks for taking the trouble to read this.
Cheers,
Brian.