procmail
[Top] [All Lists]

Re: Out of Memory errors

2002-03-15 22:40:29
On Fri, 15 Mar 2002 10:20:16 -0800, PSE wrote:

At 15:50 2002-03-15 +0700, Kevin Myers did say:
Mmm. This is the second time I've sent an item to this list,

Hmm, and people saw it the first fine, believe it or not.  I haven't 
receieved too many 128MB emails recently, nor do I run a mail server on 
just 64MB of physical memory (well, my mail servers are more than just 
email - in the olden days, I did just fine with 64MB of memory though).

Thanks for your reply. Actually the first item that I sent did happen to be
related to this same recipe, though I sent it asking for general input, not
because of this 'Out of memory' problem, which hadn't occurred at that time.
I know _I_ saw it, but as it was the first post I made in this list, I
wasn't sure that I hadn't done something wrong in the posting. 

I haven't had any 128MB emails either. I don't know why procmail is trying
to allocate so much memory. The (12) test emails that I'm using are between
2k and 2.5MB in length. 

[snip]

You omitted version and platform information, as well as *ANY* info on the 
message itself (oh, for instance, was it large?).  There was no indication 
that you were running with VERBOSE logging (including in the sandbox tests).

True. Now rectified (p 3.21 Linux kernel 2.2). I was running with VERBOSE
logging, in both the real env and the sandbox tests. I get entries like
this:

procmail: Out of memory
buffer 0: "formail -I "" | sed ':a;N;s/^\n//;s/[\n      ]\{1,\}/ /g;ta;' |
head -c 134"
buffer 1: "formail -I "" | sed ':a;N;s/^\n//;s/[\n      ]\{1,\}/ /g;ta;' |
head -c 134"
procmail: Notified comsat: "khomchalat@:**Bounced**"
From kevinmyers(_at_)unforgettable(_dot_)com  Mon Jan 14 13:04:57 2002
  Folder: **Bounced**
1200


Certain 2.4.x Linux kernels have issues with virtual memory.  I understand 
some are *VERY* picky about the exact size of the swap in relation to 
physical memory (but that's a _performance_ issue, not a showstopper).

Yes, that's partly the reason that I haven't moved up to the 2.4 kernels.

See the list archives:

        <http://www.rosat.mpe-garching.mpg.de/mailing-lists/procmail>

Search on "out of memory allocate" and you should get a fair number of hits 
to read up on.

I had already done searches (on Google) for this, and, found several
articles, but,  as I said, couldn't relate them to my particular problem.

Mar 13 16:12:37 pc procmail[1722]: Out of memory as I tried to allocate
134592524 bytes
Mar 13 16:12:43 pc procmail[1736]: Out of memory as I tried to allocate
134592524 bytes

FTR, that precise number is just a shred over 128MB, which may or may not 
exceed your combined physical and swap size (though your swap should 
probably be 2X physical).  I dunno why procmail would be trying to allocate 
so much memory unless the input message were large (which could rapidly 
happen in the event of a mail loop), or if there were a physical memory 
problem.

All the errors that I can see in the logs I have reference that number.

My swap file is 100MB, 64MB RAM. At the time I  was testing, top was
reporting something like 800k of swap being used.

However, after rebooting the server (which I hadn't done for 67 days) it
works perfectly again, with no errors at all. (Apart from a writing one
which I know about)

The last time I rebooted any of my servers was in excess of 120 days ago 
(and ironically, that was to add memory to a server just because memory is 
so darned cheap).  Prior to that, it was something on the order of a year 
(upgraded the system board), and prior to that, something in excess of 600 
days (which encounters the uptime rollover bug).

Right, so neither of us are in the habit of rebooting servers for fun, or to
clear errors. Let's not make it a pissing contest.

The output from 'top' shows that there is swap space available, the machine
has 64MB of ram.

"There is swap space available" isn't the same as "enough swap space 
available."

Open another terminal and run 'vmstat 5', then switch to the first terminal 
and run your procmail test in the sandbox.

I've done this but the trouble is, having rebooted the server, the recipe
works perfectly with no errors, so I'm not sure it's relevant. 

On the subject of which emails trigger the error, I had thought that it was
all of them.

This is interesting, though (maybe?)

When I look again, it seems to be just this syntax that triggers them:
SENDER=|$FORMAIL -b -rtzxTo:
which happens everytime in the sandbox, and when I took that out:
MESSAGE=| sed ':a;N;s/^\n//;s/[\n       ]\{1,\}/ /g;ta;' | head -c 134
which happens whenever the message is UNDER 5000 bytes.

I've seen references to memory errors in the past, but I'm not sure how to
apply those to my particular case. What should I be looking for here?

I'd start with more info about the message that triggers the error, and 
details about your specific system configuration.

top -b -n1 > topinfofile
would dump a useful process and memory snapshot for you to review.

Thank you, I'll try that next time I start getting the errors.

I still don't really know whether the problem is with procmail, hardware, or
a combination.

From your feedback, which I am grateful for, I think I can at least
eliminate my recipe as being the cause.

_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail

<Prev in Thread] Current Thread [Next in Thread>