Hi Todd,
Hamish Whittal <hamish(_at_)QEDux(_dot_)co(_dot_)za> writes:
Hi Todd
Typically, TCP window sizes go to zero when the application is not
reading from its end of the socket. The kernel's buffers fill up, and
it refuses to accept more data. That, plus the fact that you say you
don't see any 'showdots' from fetchmail suggest that fetchmail isn't
reading from the socket for some reason.
I believed the problem was with fetchmail....perhaps it still might be,
but fetchmail works fine in foreground mode. So, if I am on the console
and type fetchmail -v, then size does not remain an issue. It downloads
ALL the mails.
I am still quite confused about this manual download versus the cron-job
download. I have no clew as to why this should be the case, other than
the output is going to the console instead of a log file.
It's not clear what you mean by monitoring progress by looking at the
sendmail Q. Until fetchmail fetches the whole message, it won't hand
it to sendmail, so how do you see any progress there?
I keep a while (true) script running looking at the directory
(/var/spool/mqueue) as this was the only way I could find out what was
going on. I also did a tcpdump -i ppp0|grep for the mail server IP. As I
understand fetchmail it simply (sorry Eric for reducing it to this
simplistic term) receives POP3 (or whatever) from server and opens a
socket (I'm a layman when it comes to this lowlevel stuff) to port 25 on
sendmail. It then hands it on to sendmail. Sendmail will store the
message in the montitored directory, and distribute it as normal.
Yes, but sendmail can't put the message in that dir until it gets it
from fetchmail. Fetchmail won't give it to sendmail until it's
retrieved the whole thing. So if fetchmail isn't retrieving the mail,
you shouldn't see any progress there.
I am not sure what is in the mail, but in the /var/spool/mqueue, there
is a mail message there, and when I finally kill fetchamail (because it
does has stopped receiving the mail), the entry in the directory
disappears. So I am guessing that this is the mail message in question.
I'd suggest either attaching a debugger to fetchmail when this happens
and see what it's doing, or trying to strace it. The debugger would
be a better bet, if that works for you, but if you installed from rpm
you probably don't have the symbols.
I did [install from RPM]. I could to a strace though and would be happy
to do this if it might help [you - the developers of fetchmail]. I,
sadly, am so C rusty, I would not know my a*s from my elbow in
deciphering it.
Well, have you tried capturing fetchmail -v -v output to a file? What
does fetchmail think it's doing when this happens? Is it actually
waiting for data from the network? Maybe it's waiting for sendmail?
fetchmail -v -v and/or strace would help answer this.
OK. Will do. Should I send it to this mail list
I doubt the problem is in the kernel. It's possible, but very
unlikely. Try this for a quick check: When you see the window size
drop to 0, do a netstat -an on your end of the connection, find the
entry for the connection to your mail server, and look at the Recv-Q
column. If it's 0 or close to it, then the problem may actually be in
the kernel. If it's fairly large, as I would suspect, then it's just
that fetchmail isn't reading from the socket for some reason.
Yup, It's as you say. Recv-Q is large, and shoots up when the window
size drops to 0. I will do the fetchmail -v -v tommorrow morning to try
to troubleshoot this problem a little more.
Thanks for the help though.
Cheers
Hamish