procmail
[Top] [All Lists]

Re: Surely there's a better way for exploding digests like this

1996-11-15 00:59:28
Alan K. Stebbens <aks(_at_)anywhere(_dot_)engr(_dot_)sgi(_dot_)com> wrote in 
reply to my
query:

The following recipe is what I've created - after much sweat and toil
- to get procmail to get email from a listserver that's in digest
format and then explode it, after filtering for quoted-printable,
into individual target files (one per digest) based in the digest
number in the main `Subject:' line:

[...]

- is it possible to somehow move these KDIGEST/KFROM/DUMMY lines into
  the recipe itself so that they'll only be executed if a match has
  been made?

Of course.  Use nesting blocks. Ie:

  :0
  * some-condition-or-other
  {
    recipes which are interpreted only if the condition is true
    ...
  }

I had been experimenting with this, but found that the message would
get filtered according to any rules in there but "spill out" for
further processing.  I've since discovered - with the help of the most
excellent example you gave - that how you use the *flags* is
absolutely vital in determining what "leaves" a nesting block.

In fact, understanding the exact action of the flags turns out to be
instrumental in controlling what's going on.  There's a big difference
between `B' and `b', or their inclusion/omission, for example, to
determine what gets egrep'ed and what gets sent/piped to the action
line.

Here's how I would rewrite the above recipes:

:0
* ^(From |(Cc|To):(.*[^a-zA-Z0-9._-])?)linux-kernel-digest
* ^Subject:.*#\/[0-9]+
{
  KDIR=kernel # don't need $MAILDIR in front, since we are in $MAILDIR
  KDIGEST=$MATCH      # the digest number

  :0fW                # do mime filtering if there is content-type embedded
  * ^Content-Type:    
  | email -f 2 -l 2 -C ISO-8859-1 -F RFC822 -H 8bit -T 8bit

  :0fBW               # do signature cleanup if needed
  * ^(-- $|------------------------------)
  | sed   -e 's+^- ++' \
            -e 's+------------------------------++'

  :0fBW               # do PGP cleanup if needed
  * ^-+(BEGIN|END) PGP (SIG(NATURE|NED MESSAGE)|PUBLIC KEY BLOCK)-+ *$
  | sed -e '/BEGIN PGP SIGNED MESSAGE/d' \
        -e '/BEGIN PGP PUBLIC KEY BLOCK/,/END PGP PUBLIC KEY BLOCK/d' \
        -e '/BEGIN PGP SIGNATURE/,/----------------------------/d'

  :0:         # finally, burst the digest and deposit
  | formail -i"Sender: 
owner-linux-kernel-digest(_at_)vger(_dot_)rutgers(_dot_)edu" -Yds >> $KDIGEST
}

Pure Poetry!  :-)

Works just great.  Well, almost works :)  I still had to cope with
creating a subdirectory and a destination filename if they did not
already exist - easily done (with your help).  And the `email' program
called there should really be `emil'.  Small matters... the important
thing is that the "penny has finally dropped" :-)

I have modified this recipe to use with the linux-announce digest that
I also get:

[ ~/.procmailrc ]

# do mime filtering for all incoming Quoted-Printable messages
#
:0fW
* ^Content-Transfer-Encoding: *QUOTED-PRINTABLE
# This is more specific than `* ^Content-Type:'
# I can handle most mime, but I just *hate* QP :)
| emil -f 2 -l 2 -u -C ISO-8859-1 -F RFC822 -H 8bit -T 8bit
#
        :0 Afhw
        | formail -I "Content-Transfer-Encoding: 8bit"
[...]

# owner-Linux-Announce(_at_)senator-bedfellow(_dot_)MIT(_dot_)EDU
# Linux-Announce-Request(_at_)NEWS-DIGESTS(_dot_)MIT(_dot_)EDU
#
:0
* ^(From |(Cc|To):(.*[^a-zA-Z0-9._-])?)linux-announce
* ^Subject:.*#\/[0-9]+
{
  ADIR=announce
  NUMBER=$MATCH                         # the digest number
  ADIGEST=$ADIR/digest.$MATCH           # the file to save it to
  X=`test -d $ADIR || mkdir $ADIR`
  X=`test -f $ADIGEST || touch $ADIGEST`
#
# enable these two lines and disable the rest of the recipe to get
# each digest in simple unfiltered/unexploded format, one digest per file.
#  :0
#  $ADIGEST
#
   :0Hh                                 # extract the "From " line
   FROM=|sed -n 1p
# 
   :0fBW                                # PGP cleanup
   * BEGIN PGP SIGN
   | sed -e '/BEGIN PGP SIGNED MESSAGE/d' \
         -e '/BEGIN PGP SIGNATURE/,/----------------------------/d'
# 
   :0fBW                                # signature/digest-escaping cleanup
   * ^(-- $|------------------------------)
   | sed        -e 's+^- ++' \
        -e 's+------------------------------++'
# 
   :0Wc:                                # burst the digest
   | sed -e '/^\*\* FOR YOUR REFERENCE \*\*/,$d' \
   | formail -i"Sender: 
owner-Linux-Announce(_at_)senator-bedfellow(_dot_)MIT(_dot_)EDU" -Yds \
   >> $ADIGEST
# 
   :0Bhb:                               # add the tailing message
   * ^\*\* FOR YOUR REFERENCE \*\*
   | sed -n -e '/^\*\* FOR YOUR REFERENCE \*\*/,$p' \
   | formail -f -a"$FROM" \
                -A"To: 
Linux-Announce(_at_)senator-bedfellow(_dot_)MIT(_dot_)EDU" \
                -A"Subject: ********* End of Linux-Announce Digest #$NUMBER 
*********" \
                -A"From: Digestifier 
<Linux-Announce-Request(_at_)senator-bedfellow(_dot_)MIT(_dot_)EDU>" \
   >> $ADIGEST
#
}

Hacking around to get the trailing lines from the digest message
cleanly added as a separate message (to clearly delineate each
exploded digest) proved to be a very instructive experience :-)

To get my procmail library, send me an email with the subject of "send
procmail library", or browse my web page under the mail link.
___________________________________________________________
Alan Stebbens <aks(_at_)sgi(_dot_)com>      http://reality.sgi.com/aks

Again, many thanks for your help, it was invaluable.

The power of procmail is awesome - it's one mighty fine piece of
software.  I'm grateful for it.

Cheers
Tony

<Prev in Thread] Current Thread [Next in Thread>