At 23:37 2003-03-30 -0800, Jeff Orrok wrote:
I have a fairly involved suite of procmailrc's for a client that processes
all of the email leads that he receives from visitors to various websites.
Everything seems to be working ok, but today when I was testing in preparation
for upgrading, I was horrified to discover that it was taking almost 8 minutes
to process 8 messages in my test file. I'm not sure when it started doing
this, since I proceeded to regress my changes, and even pulled his current
copy
over for comparison, and every version seems to be doing the same thing.
I can't believe that anything I'm doing would take up that much time, but I
guess I've got to face the music. So can anyone point me in the direction of
some performance analysis utilities that I can use to track down my problem?
1. migrate the complete recipes into a sandbox. I have the sandbox script
INCLUDERC my actual filter, so the sandbox framework itself remains
unchanged over time (except for a revision here and there).
With a null test filter, you could run:
time formail -s procmail -m sandbox.rc < somemailboxfile
several times to establish a benchmark for the overhead of the sandbox
itself and (potentially) cache the messages. Then, replace the test rcfile
with the contents of your RC file and re-run it a few more times, and
you'll have an indication on how much slower your recipes are.
There's a lot of other stuff you could do, but I'd start with getting a
better handle on overhead v. specifics of your script. Then, you can look
at the zillions of shell operations, unnecessarily repeated shells, and
wasteful pipelines.
FTR, grep (the program) in 'word' mode is notoriously slow and is a major
memory hog, esp if you're matching a large message. So slow in fact, that
I rolled my own "megagrep" utilizing an AVL tree and specialty message
parser to optimize matching a message against a 3.5MB list of entries (and
several smaller lists too).
---
Sean B. Straw / Professional Software Engineering
Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
Please DO NOT carbon me on list replies. I'll get my copy from the list.
_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail