procmail
[Top] [All Lists]

Re: Surely there's a better way for exploding digests like this

1996-11-13 11:43:01
    > The following recipe is what I've created - after much sweat and toil
    > - to get procmail to get email from a listserver that's in digest
    > format and then explode it, after filtering for quoted-printable,
    > into individual target files (one per digest) based in the digest
    > number in the main `Subject:' line:
    > 
    > 
    > KDIR=$MAILDIR/kernel
    > KDIGEST=$KDIR/digest.`formail -x Subject:| cut -d\# -f2`
    > KFROM=`formail -x Subject:|cut -d\  -f4`
    > DUMMY=`test ! X"$KFROM" = X"linux-kernel-digest" || test -d $KDIR || 
mkdir $KDIR`
    > DUMMY=`test ! X"$KFROM" = X"linux-kernel-digest" || test -f $KDIGEST || 
touch $KDIGEST`
    > :0fW: $KDIGEST.lock
    > * (^From[^:]|^Cc:|^To:).*linux-kernel-digest
    > | emil -f 2 -l 2 -C ISO-8859-1 -F RFC822 -H 8bit -T 8bit \
    > | formail -i"Sender: 
owner-linux-kernel-digest(_at_)vger(_dot_)rutgers(_dot_)edu" -Yds \
    > | sed   -e 's+^- ++' \
    >         -e 's+------------------------------++' \
    >         -e '/BEGIN PGP SIGNED MESSAGE/d' \
    >         -e '/BEGIN PGP PUBLIC KEY BLOCK/,/END PGP PUBLIC KEY BLOCK/d' \
    >         -e '/BEGIN PGP SIGNATURE/,/----------------------------/d' \
    > >> $KDIGEST
    > 
    > 
    > Now that's really ugly, but I tinkered with lots of different
    > variations and what I have there is the only thing that I got to work.
    > 
    > An example of the Subject line is:
    > 
    > Subject:   linux-kernel-digest V1 #595
    > 
    > So the idea of the $KDIGEST variable is to extract the number from
    > this line and use it as a "volume number" as part of the target name
    > for the output of the filter.  All messages in the digest get nicely
    > exploded into files such as:
    > 
    > ~/mail/kernel/digest.595
    > 
    > The $DUMMY lines are there simply to create the directory and the file
    > if they don't already exist - or else procmail will simply skip this
    > whole thing altogether because the filter fails due to the shell's
    > complaint about a non-existant file.  (It's bash, and
    > redirect/appending with `>>' to a non-existant file usually works ok
    > to create it --- but strangely, not in this case).
    > 
    > I tried doing something like
    > 
    > | (touch $KDIGEST ; cat) \
    > 
    > at the start of the filtering, but it seems that the target file must
    > exist *before* the filtering begins (from what turns up in the log,
    > it's most likely a requirement of the shell rather than procmail).
    >
    > `emil' is a filter for doing all sorts of mime-conversions; in this
    > case it turns quoted-printable into 8bit/rfc822 - and it works very
    > well.  The rest of the filter is fairly self-explainatory.
    > 
    > The KDIGEST/KFROM and DUMMY lines get executed for *every* message
    > which hasn't been "picked off" by previous rules.  (So I have this
    > recipe near the end of my ~/.procmailrc).  This seems to be very
    > inefficint to me, especially since they only need to be done if the
    > match for this rule has been made.
    > 
    > My question:
    > 
    > - is it possible to somehow move these KDIGEST/KFROM/DUMMY lines into
    >   the recipe itself so that they'll only be executed if a match has
    >   been made?

Of course.  Use nesting blocks. Ie:

  :0
  * some-condition-or-other
  {
    recipes which are interpreted only if the condition is true
    ...
  }

    > - is there any way to get procmail to carry out actions (like touch
    >   files) within (or outside) filter sequences without then having to
    >   be part of the filtering process?  Or is this "DUMMY" method the way
    >   things like this are generally achieved?

There are two ways to run shell commands only for their actions and not
their output: either in an evaluated string (`commands`), or by using a
action on a "copy" recipe.

  X=`some-shell-command args blah blah`

or

  :0c
  |some-shell-command args blah blah

or
  :0c
  VAR=|some-shell-command args blah blah

The last recipe captures any STDOUT into the variable VAR.

Here's how I would rewrite the above recipes:

    > KDIR=$MAILDIR/kernel
    > KDIGEST=$KDIR/digest.`formail -x Subject:| cut -d\# -f2`
    > KFROM=`formail -x Subject:|cut -d\  -f4`
    > DUMMY=`test ! X"$KFROM" = X"linux-kernel-digest" || test -d $KDIR || 
mkdir $KDIR`
    > DUMMY=`test ! X"$KFROM" = X"linux-kernel-digest" || test -f $KDIGEST || 
touch $KDIGEST`
    > :0fW: $KDIGEST.lock
    > * (^From[^:]|^Cc:|^To:).*linux-kernel-digest
    > | emil -f 2 -l 2 -C ISO-8859-1 -F RFC822 -H 8bit -T 8bit \
    > | formail -i"Sender: 
owner-linux-kernel-digest(_at_)vger(_dot_)rutgers(_dot_)edu" -Yds \
    > | sed   -e 's+^- ++' \
    >         -e 's+------------------------------++' \
    >         -e '/BEGIN PGP SIGNED MESSAGE/d' \
    >         -e '/BEGIN PGP PUBLIC KEY BLOCK/,/END PGP PUBLIC KEY BLOCK/d' \
    >         -e '/BEGIN PGP SIGNATURE/,/----------------------------/d' \
    > >> $KDIGEST

    :0
    * ^(From |(Cc|To):(.*[^a-zA-Z0-9._-])?)linux-kernel-digest
    * ^Subject:.*#\/[0-9]+
    {
      KDIR=kernel       # don't need $MAILDIR in front, since we are in $MAILDIR
      KDIGEST=$MATCH    # the digest number

      :0fW              # do mime filtering if there is content-type embedded
      * ^Content-Type:  
      | email -f 2 -l 2 -C ISO-8859-1 -F RFC822 -H 8bit -T 8bit

      :0fBW             # do signature cleanup if needed
      * ^(-- $|------------------------------)
      | sed   -e 's+^- ++' \
              -e 's+------------------------------++'

      :0fBW             # do PGP cleanup if needed
      * ^-+(BEGIN|END) PGP (SIG(NATURE|NED MESSAGE)|PUBLIC KEY BLOCK)-+ *$
      | sed -e '/BEGIN PGP SIGNED MESSAGE/d' \
            -e '/BEGIN PGP PUBLIC KEY BLOCK/,/END PGP PUBLIC KEY BLOCK/d' \
            -e '/BEGIN PGP SIGNATURE/,/----------------------------/d'

      :0:               # finally, burst the digest and deposit
      | formail -i"Sender: 
owner-linux-kernel-digest(_at_)vger(_dot_)rutgers(_dot_)edu" -Yds >> $KDIGEST
    }

    > Finally -
    > 
    > - I'm slowly switching over to use mh, so is there any easy way to get
    >   mail digests to be easily exploded into directories as separate
    >   files in numbered sequences?  (Or is it easier to just use
    >   /usr/lib/mh/rcvstore as a filter to do this?)

Yes.  Use mh-style multiple folders on the filing action: (see procmailrc).

  :0
  mhfolder1/. mhfolder2/. mhfolder3/.

If you get my procmail library, you'll see some recipe files called
"pf-<mumble>.rc", which perform prefiling using either normal or
MH-style folders.

To get my procmail library, send me an email with the subject of "send
procmail library", or browse my web page under the mail link.

___________________________________________________________
Alan Stebbens <aks(_at_)sgi(_dot_)com>      http://reality.sgi.com/aks

<Prev in Thread] Current Thread [Next in Thread>