Author: era eriksson <era(_at_)iki(_dot_)fi>
Original-Date: Mon, 13 Oct 1997 09:09:34 +0300 (EET DST)
Message-ID:
<199710130609(_dot_)JAA19818(_at_)kontti(_dot_)Helsinki(_dot_)FI>
I still think it's kind of embarrassing that Procmail would not be
able to cope with big messages. If the normal local MDA could handle
it, why can't Procmail?
Procmail blew chunks all over a 9meg email I received this weekend.
Sucked all available CPU and swapspace (about 250megs and a P-133).
Got lots of duplicate emails and lost emails in the resultant disk crash(es)
which required manual fsck-ing.
I had got this also on my FreeBSD machine with 64MB RAM and 250MB
swap. No message bigger than 5-8 MB was possible. However, as I was
told from the vm and malloc experts of FreeBSD, that this is due
to the silly way how procmail reads messages: It always reallocs
memory in 16k chunks until the message fits into (virtual) memory.
With this info, I made a quick and dirty hack to procmail, and
now it delivers 64MB messages on the same machine and that really
quick. It now allocs 4MB chunks which, on a modern system, only
get used if there is actually written to it.
*** src/pipes.c.ORI Mon Apr 28 02:27:47 1997
--- src/pipes.c Fri Sep 19 13:23:18 1997
***************
*** 234,239 ****
--- 234,240 ----
char*readdyn(bf,filled)char*bf;long*const filled;
{ int i;long oldsize;
+ long fetched = *filled;
oldsize= *filled;
goto jumpin;
do
***************
*** 243,249 ****
if((size_t)*filled>=(size_t)(*filled+BLKSIZ))
lcking|=lck_MEMORY,nomemerr();
#endif
! bf=realloc(bf,*filled+BLKSIZ); /* dynamically adjust the buffer size
*/
jumpback:;
}
while(0<(i=rread(STDIN,bf+*filled,BLKSIZ))); /* read
mail */
--- 244,253 ----
if((size_t)*filled>=(size_t)(*filled+BLKSIZ))
lcking|=lck_MEMORY,nomemerr();
#endif
! if( fetched - *filled < BLKSIZ ) {
! fetched += 4194304;
! bf=realloc(bf,fetched); /* dynamically adjust the buffer size */
! }
jumpback:;
}
while(0<(i=rread(STDIN,bf+*filled,BLKSIZ))); /* read
mail */
-Andre