Re: Symetrical Multiprocessing w/ Procmail and Spamassassin

At 23:33 2003-08-06 -0700, mindfuq(_at_)comcast(_dot_)net wrote:
> My pentium 166 just crawls when it comes to procmail, and I think
> Spamassassin is part of the hog.
Well, Mr. Fuq, by current standards, a Pentium 166 crawls anyway.  It 
probably doesn't have all that much memory either, which compounds matters.
Come to think of it, a Pentium-166 crawled six years ago.  I built a Dual 
PPro-200 box in what, June of 1996 or so, so that's seven years, and even 
that was one processor generation past the Pentium.  I'd have to say that a 
166MHz processor hasn't done any screaming in quite some time.
That's not to say that the operator of a system that old hasn't done some 
screaming at the console.
If you felt things were dogging along, perhaps you should have considered 
upgrading to something resembling _current_ hardware?
> So I bought a dual P2 400.
Congrats.  I predict you'll be laying down rubber with it soon.  I'd 
caution against the ground effects and chrome wheel trim though.
> When I do a TOP, I can see that everything is balanced between the two 
> processors, but when I run procmail, there is only one procmail process 
> and one spamassassin process.  This isn't what I expected.
What, you expect that when you run procmail, it should fire up multiple 
processes just for kicks?  This is all fine and dandy for server daemons - 
Apache and SQL, but other programs simply don't run additional processes 
unless they need them right then and there.
I can't rightly think of anything in procmail that would really benefit 
from threading anyway - the whole thing is pretty darn linear.  A threaded 
regexp library might be nifty, but is beyond the scope of procmail development.
I might also point out - not that it has a whit to do with procmail, which 
isn't threaded - but depending on the thread library used and the OS it is 
used on, threads may not appear as discrete processes in the process list, 
and therefore, a controlling process running multiple threads may still 
appear to be one process in ps/top - conversely, some libraries on some OS' 
may have threads appear as processes, and say, an SQL daemon may appear to 
be running in excess of 100 times, when there's really only one daemon 
running, with many threads at its disposal.
> When I ran procmail on my original single cpu machine, I always saw
> multiple processes,
Which would have been the result of multiple concurrent emails, or of a 
number of 'c' flags in your recipes.
>  which didn't do a lot of good w/ one processor.
> Now after switching to a 2 processor machine, I was expecting double
> the procmail speed.  Why does it only run one process at a time?
Lessee, you have two processors, each approaching three times the clock 
speed of your original processor.  I imagine that overall, you've got to be 
seeing about 4-5 times the overall system speed.  How is it that procmail 
doesn't seem to finish things quicker for you?
Note that the _other_ processor, even if a single instance of procmail 
isn't utilizing it, can still be running some other process which would 
normally bog down the first processor, which means that procmail has more 
opportunity for CPU utilization when the system is loaded.
Also, if individually, the procmail processes can complete quicker (since 
either one of the CPUs is much faster, and with the CPU boost, I suspect 
you also have a memory boost to the machine), rather than lingering around, 
you may not see multiple processes because by the time a second email 
message has arrived (over your comparatively slow network connection) to be 
processed, the first process has managed to complete.  On your old system, 
lack of memory resources can lead to excessive paging, which slows 
processes down to a crawl, which in turn increases the chances that the MTA 
will be firing off a concurrent process for a second message before the 
first message you recieved has completed processing (and each concurrent 
process takes progressively more time because there are fewer system 
resources).
Pick up a good book on system administration, or even a book such as 
_Programming_With_UNIX_Threads_ (ISBN 0-471-13751-0) by Charles J. Northrup 
if you want a better understanding of threads.
In the meantime, keep in mind that procmail doesn't USE threading.

---
 Sean B. Straw / Professional Software Engineering

 Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
 Please DO NOT carbon me on list replies.  I'll get my copy from the list.


_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail