procmail
[Top] [All Lists]

Re: Procmail flaky??

2002-04-26 00:50:19
At 11:56 2002-04-25 -0700, Edward Huang did say:
30 employees. now procmail has been implemented as a gateway for atmel.com with a volume of over 20,000 emails/day getting a lot of false filtering

After careful review of the material you presented, I suspect that when you were processing mail for 30 employees you may have been operating with a smaller set of rules which didn't contain errors, and/or, in your switch to filtering email for the entire organization, your filters were then exposed to a larger variety of messages which exposed the bad logic in some of the filters.

ONE EXAMPLE:

(which lacks the matching script - procmail executes SCRIPTS, not log excerpts)

[kersnip]

procmail: Couldn't determine implicit lockfile from "/usr/lib/sendmail"
procmail: Locking ".lock"

Lemme guess - you have the locking flag ':' at the end of your flags line, but as the above message would indicate, you are NOT delivering to a FILE. Guess what? You don't need to lock, and if for some reason you did, you shoud provide an explicit lockfilename (after the colon).

procmail: Executing "/usr/lib/sendmail,-oi,spam"
procmail: Assigning "LASTFOLDER=/usr/lib/sendmail -oi spam"

Okay, looks successful like.

procmail: Unlocking ".lock"
 Subject: sweet
Folder: /usr/lib/sendmail -oi spam 575

Whoohoo, looks fine.

procmail: [5892] Thu Apr 25 11:20:45 2002
procmail: Assigning "EOL=
"
procmail: Assigning "LOG=
"




the word sweet only appears in the SUBJECT of the email
the rest of the email is blank.   yet, Procmail's ^FROM
receipe matched it.

Let's be clear here - procmail doesn't have a ^FROM macro (i.e. there's nothing approximating ^TO or ^TO_). Since you managed to NOT provide the recipe, and only provided the log, let me reconstruct what I can from your log - pardon me if it's incomplete, as your omission of it is a rather critical one:


:0:
* ^From:.*Free Money|JDR Direct Offers|Web Hosting|Channels|Free Recruitment|Equalamail|Money Maker|Shirts Express|Lifestyle|SpecialOffers|Aut o Financing|downloads|PennyHaven|SmallCap|Scholarships|Payment Center|FoundMoney|MonsterHut|Joke|Child Support|EMailHello|TopOffers|Free Stuff|uBid|GetRichNow|MyPre|Sweet
! spam


The first thing here is that since we're not delivering to a FOLDER, or to a script which might be outputting to a database or otherwise be restricted to one concurrent invocation, there's no need for the ':' flag, and it should therefore be omitted - you shouldn't just toss the flag in there on every recipe.

FTR, if these filters are running from within an /etc/procmailrc script, you might just dumo the messages directly into a spam mailfolder, rather than _delivering_ them to the spam user. It'd waste less system resources. Of course, if you dump it into a file, you WOULD want file locking...


Next, consider the two following statements:

* ^From:.*Free Money|JDR Direct Offers

and

* ^From:.*(Free Money|JDR Direct Offers)

Examine them closely - they're VERY different.

Hint: the OR '|' operator matches within the extents of the paren -- and if no parens, then it matches to the extents of the EXPRESSION. Thus, YOUR expression matches:

        ^From:.*Free Money
        OR
        JDR Direct Offers               (ANYWHERE within the headers)
        OR
        (any of the other keywords, ANYWHERE within the headers)

You might want to spend some quality time becoming intimate with the procmail man pages. 'man procmailrc' and 'man procmailex' for starters.

this and many other cases lead me to believe that Procmail 3.15 is either or both:

I respectfully submit that you have "many other cases" of similar incorrectly composed expressions. Before trashing too much more mail, you should carefully review your mail filters and FIX them.

a) not confining its egrep to THAT ^SUBJECT or ^FROM ***LINE***

No, *YOU* are providing procmail with a BAD regular expression - one that even a human can parse as being incorrect for the situation you're using it in. Don't be so quick to blame procmail for your mistakes.

GIGO. Garbage In, Garbage Out. If you give procmail bad instructions, don't be surprised if it doesn't do what you expected.

b) is searching entire header or body even though I tried
forcing :0H: (and the default is supposed to be Header Only)

Pardon? You indicate that the keyword of "Sweet" appeared in the Subject. A wee little reminder: Subject is a HEADER. If H is implied, there is no reason to specify it, unless you were trying to match Body and Headers: BH. I suspect you're throwing flags at a problem hoping they'll fix your script.

You might want to check out my sandbox info -- see my .sig.

---
 Sean B. Straw / Professional Software Engineering

 Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
 Please DO NOT carbon me on list replies.  I'll get my copy from the list.

_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail

<Prev in Thread] Current Thread [Next in Thread>