procmail
[Top] [All Lists]

Re: maybe I'm missing something (Re: splitting digests, catching duplicates )

1996-12-06 00:31:18
    > Responding To:    "Alan K. Stebbens" 
<aks(_at_)anywhere(_dot_)engr(_dot_)sgi(_dot_)com>
    > Original Date:    Thu, 05 Dec 1996 18:56:02 -0800
    > Message-ID:       
<199612060256(_dot_)SAA24422(_at_)anywhere(_dot_)engr(_dot_)sgi(_dot_)com>
    > 
    > 
    > # from .procmailrc
    > :0
    > * ^From: Antigone List Manager <listmgr(_at_)antigone(_dot_)com>
    > {
    >   :0c
    >   * ^Subject: next-announce
    >   NEWSGROUP=comp.sys.next.software

This is wrong.  It should be written:

    :0
    * ^Subject: next-announce
    { NEWSGROUP=comp.sys.next.software }

The 'c' flag is needed only to keep a "delivering" recipe from becoming
a final action.  It can also be used to cause a fork on a nested block.
So, if you are using a recipe to conditionally set a flag, don't use
a 'c' option, or you will get two procmail processes flowing past the
closing brace.

Also, keep spaces around both braces.

    >   :0c
    >   * ^Subject: next-bugs
    >   NEWSGROUP=comp.sys.next.bugs
    >
    >   :0c
    >   * ^Subject: next-hardware
    >   NEWSGROUP=comp.sys.next.hardware
    >
    >   :0c
    >   * ^Subject: next-marketplace
    >   NEWSGROUP=comp.sys.next.marketplace
    > 
    >   :0c
    >   * ^Subject: next-misc
    >   NEWSGROUP=comp.sys.next.misc
    > 
    >   :0c
    >   * ^Subject: next-software
    >   NEWSGROUP=comp.sys.next.software
    > 
    >   :0c
    >   * ^Subject: next-sysadmin
    >   NEWSGROUP=comp.sys.next.sysadmin
    > 

When you have a series of mutually exclusive recipes, you
should use the 'E' flag: it makes procmail more efficient.

    :0
    * ^Subject: next-announce
    { NEWSGROUP=comp.sys.next.software }
    :0E
    * ^Subject: next-bugs
    { NEWSGROUP=comp.sys.next.bugs }
    :0E
    * ^Subject: next-hardware
    { NEWSGROUP=comp.sys.next.hardware }
    :0E
    * ^Subject: next-marketplace
    { NEWSGROUP=comp.sys.next.marketplace }
    :0E
    * ^Subject: next-misc
    { NEWSGROUP=comp.sys.next.misc }
    :0E
    * ^Subject: next-software
    { NEWSGROUP=comp.sys.next.software }
    :0E
    * ^Subject: next-sysadmin
    { NEWSGROUP=comp.sys.next.sysadmin }

Another way to accomplish this is to create a table of subjects
and corresponding newsgroups in a file (I'll call it "next-groups"):

  next-groups:
    next-announce       comp.sys.next.software
    next-bugs           comp.sys.next.bugs
    next-hardware       comp.sys.next.hardware
    next-marketplace    comp.sys.next.marketplace
    ... # and so on

Then, a recipe to use this file might look like:

    NEWSGROUP=|formail -zxSubject: | xargs -i% awk '$1=="%"{print $2}' 
next-groups

If you don't like using xargs and an extra pipe, you can have the shell
escape-pipe:

    NEWSGROUP=|awk '$1=="'`formail -zxSubject:`'"{print $2}' next-groups

This works only because formail reads STDIN, as part of the backquote
invocation, before the shell invokes "awk".  

The second method is more concise, but requires exact quoting.

    :0fh                # insert the newsgroup header if the subject matches
    * NEWSGROUP ?? .
    | formail -I"X-TjL-Newsgroup: $NEWSGROUP"

This extracts the subject, and then use the subject to lookup, in the
first column, a corresponding newsgroup from the second column in the
file "next-groups".

    >   :0chi >   |/bin/echo "newsgroup is $NEWSGROUP" >/dev/console

If you are testing to see if your recipe works, you can use assignments to LOG:

    LOG="newsgroup is $NEWSGROUP
    "
Note the embedded newline.

If you really wish to see output on the console, use the "i" flag
(to tell procmail that you are not going to read STDIN), and the 'c'
flag (to tell procmail that your are not performing a final
delivery action).

    :0 ci
    |echo "newsgroup in $NEWSGROUP" >/dev/console

Again, with 'f', you don't need the 'c' flag.  So, write this:

    >   :0fhc
    >   | formail -I "X-TjL-Newsgroups: $NEWSGROUP"

as this:
    
    :0fh
    |formail -I"X-TjL-Newsgroups: $NEWSGROUP"

And this recipe should be invoked only if the input is really a digest:

    >   :0: proc-filter.lock
    >   |formail +1 -des procmail -m /usr/local/bin/proc-filter
    > 
    > }


    >   # proc-filter is:
    > 
    > #!/usr/local/bin/procmail -m
    > 
    > # whatever recipes you like, such as these...
    > 
    > SHELL=/bin/sh

Assigning shell is always a good idea!

    > GROUP=`formail -xX-TjL-Newsgroups:`
    > 
    > :0c
    > |/bin/echo "group is >$GROUP<" >/dev/console
    > 
    > :0Wh:             # check the message-ids
    > * ^Message-Id:
    > | formail -D 8192 .msgids
    > 
    > :0E                       # else, check for md5 checksums
    > { :0b
    >   SUM=|md5
    >   LOCKFILE=.checksums.lock
    >   :0Wih
    >   | fgrep -s "$SUM" .checksums
    >   JUNK=`(tail -8000 .checksums; echo "$SUM")>.checksums.new;\
    >          mv .checksums.new .checksums`
    >   LOCKFILE
    >   :0fhc
    >   |formail -I"X-Checksum: $SUM"
    >   
    >   :0                      
    >   |appnmail Computer-Stuff/$GROUP
    > 
    >   # appnmail is a NeXTStep thing that appends messages to the 
    >   #       NeXT mailbox.  That part is fine, it is the "$GROUP"
    >   #       that is not being set to anything
    > 
    > }
    > 
    > 
    > 
    > 
    > Verbose log output:
    > 
    > procmail: Assigning "LOG=
    > "
    > 
    > procmail: Assigning "LOGABSTRACT=yes"
    > procmail: Executing "formail,-xSubject:"
    > procmail: Assigning "SUBJECT= next-misc 1117"
    > procmail: [1062] Fri Dec  6 00:29:36 1996
    > procmail: Executing "formail,-xDATE:"
    > procmail: [1062] Fri Dec  6 00:29:36 1996
    > procmail: Assigning "DATE= Wed, 4 Dec 1996 04:25:20 -0800 (PST)"
    > procmail: Executing "formail,-rtzxTo:"
    > procmail: Assigning "SENDER=luomat(_at_)nerc(_dot_)com"
    > procmail: [1062] Fri Dec  6 00:29:37 1996
    > procmail: Executing "formail,-xFrom:"
    > procmail: [1062] Fri Dec  6 00:29:37 1996
    > procmail: Assigning "FROM2= Antigone List Manager 
<listmgr(_at_)antigone(_dot_)com>"
    > procmail: Assigning "NAMES=/Users/luomat/.procmail/email-names.txt"
    > procmail: Executing "formail,-rt"
    > procmail: Assigning "NAME=To: luomat(_at_)nerc(_dot_)com
    > Subject: Re: next-misc 1117
    > References: <199612041229(_dot_)EAA12878(_at_)lists1(_dot_)best(_dot_)com>
    > In-Reply-To: 
<199612041229(_dot_)EAA12878(_at_)lists1(_dot_)best(_dot_)com>"
    > procmail: Assigning "DIGESTS=Computer-Stuff/NeXT-Digests/"
    > procmail: Executing "appnmail,-r,Misc/backups"
    > procmail: [1062] Fri Dec  6 00:29:38 1996
    > procmail: Assigning "LASTFOLDER=appnmail -r Misc/backups"
    > procmail: No match on "^From PopOver"
    > procmail: No match on "^^X-Nvlenv-01Date-Posted:"
    > procmail: No match on "^Subject: PPP_IP"
    > procmail: No match on "-----BEGIN PGP MESSAGE-----"
    > procmail: No match on "^Subject:.(*ubscribe|virus|*virus|*try  
    > this|*money)"
    > procmail: Executing  
    > "grep,-iw,luomat(_at_)nerc(_dot_)com,/Users/luomat/.procmail/spammers.txt"
    > procmail: [1062] Fri Dec  6 00:29:40 1996
    > procmail: Program failure (1) of "grep"
    > procmail: No match on "grep -iw luomat(_at_)nerc(_dot_)com  
    > /Users/luomat/.procmail/spammers.txt"
    > procmail: No match on "^From: listmgr(_at_)antigone(_dot_)com"
    > procmail: Match on "^From: Antigone List Manager 
<listmgr(_at_)antigone(_dot_)com>"
    > procmail: No match on "^Subject: next-announce"
    > procmail: No match on "^Subject: next-bugs"
    > procmail: No match on "^Subject: next-hardware"
    > procmail: No match on "^Subject: next-marketplace"
    > procmail: Match on "^Subject: next-misc"

The error message below makes it clear what is happening: procmail didn't
make an assignment to NEWSGROUP; it tried to *write* to a filename of
"NEWSGROUP=comp.sys.next.misc".  This is because the braces were
missing around the variable assignment, so procmail assumed that
the entire string was a filename.

    > procmail: Assigning "LASTFOLDER=NEWSGROUP=comp.sys.next.misc"
    > procmail: Opening "NEWSGROUP=comp.sys.next.misc"
    > procmail: Acquiring kernel-lock
    > procmail: No match on "^Subject: next-software"
    > procmail: No match on "^Subject: next-sysadmin"
    > procmail: Executing "/bin/echo "newsgroup is $NEWSGROUP" >/dev/console"
    > procmail: Assigning "LASTFOLDER=/bin/echo "newsgroup is $NEWSGROUP"  
    > >/dev/console"
    > procmail: [1062] Fri Dec  6 00:29:41 1996
    > procmail: Executing "formail,-I,X-TjL-Newsgroups: "
    > procmail: [1062] Fri Dec  6 00:29:41 1996
    > procmail: Locking "proc-filter.lock"
    > procmail: Executing  
    > "formail,+1,-des,procmail,-m,/usr/local/bin/proc-filter"

This error below occurred because echo didn't read STDIN, and you didn't
tell procmail to ignore it with the 'i' flag.

    > procmail: Error while writing to "/bin/echo "group is >$GROUP<"  
    > >/dev/console"
    > procmail: [1062] Fri Dec  6 00:29:47 1996
    > procmail: Assigning "LASTFOLDER=formail +1 -des procmail -m  
    > /usr/local/bin/proc-filter"
    > procmail: Unlocking "proc-filter.lock"
    > >From luomat  Fri Dec  6 00:29:35 1996
    >  Subject: next-misc 1117
    >   Folder: formail +1 -des procmail -m /usr/local/bin/proc-filter   
    >    8231
    > procmail: Notified comsat: "luomat@:formail +1 -des procmail -m  
    > /usr/local/bin/proc-filter"

Good luck.
___________________________________________________________
Alan Stebbens <aks(_at_)sgi(_dot_)com>      http://reality.sgi.com/aks

<Prev in Thread] Current Thread [Next in Thread>
  • Re: maybe I'm missing something (Re: splitting digests, catching duplicates ), Alan K. Stebbens <=