procmail
[Top] [All Lists]

Re: Some ideas to improve performance

2004-11-24 07:21:21
Toen wij Dallman Ross kietelden, kwam er dit uit:
Ruud H.G. van Tol:
mailarch(_at_)xy1(_dot_)org:

The file.mbox contains more than 10 000 messages!
This means that formail will pipe(), fork() and exec()
procmail 10 000 times!!!
It's very slow!

There are easy ways to improve on that. One way is to use
lockfiles that have the seconds-part of the Date-header in
the filename (which would limit it to 60 procmails). In the
same way you can limit it to 10 or 600 procmails, or whatever.

Ruud, can you explain more about that?

To prevent 10K procmails fighting for resources, you can use 
lockfiles to prevent more than a limited number of procmails 
running at the same time. 


Anyway, one could use formail or readmsg to extract a set
of messages -- say, 1000 at a time -- and run the main process
on those smaller files.

That could well be a simpler way. But a full-text-search needs 
access (once?) to all the messages.

Maybe mharc supports maildir? You might need a small blocksize, 
or a filesystem that packs small files together in a block, 
to limit the diskusage.

-- 
Grtz, Ruud

____________________________________________________________
procmail mailing list   Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail