procmail
[Top] [All Lists]

Re: sending usenet messages

1996-08-19 18:42:22
I wrote:

1b. Alternatively, the freeware 'newsdigest' program maintains a .newsrc
file and create a mailable digest of the new articles, which can be
forwarded directly to the mirror list.

Ari writes:

You mean there is such a thing that will get news articles for you???

I would love to get newsarticles with certain subjects in the title or by
author. I would pay for it, too. Is there such a thing?

Brian writes:

I would also be interested in such a product.  Would it be compatible with 
INN on Solaris2.5?

My log says that I found Newsdigest (v. 2.1) in two places

    qiclab.scn.rain.com opened at Sat Jan  7 00:19:08 1995
        -> "/pub/news/Newsdigest-2.1.shar.Z" received, 1.62 K/s
    
    liasun3.epfl.ch opened at Tue Feb 28 17:09:46 1995
        -> "/pub/util/Newsdigest-2.1.tar.gz" received, 4.75 K/s

It's a small C program, and it does have filtering capability (implemented
at at the client end).  As far as I know works with any nntp server.  I'm
attaching the man page.

P-)
         ___o        -     o             Peter Marks 
<marks(_at_)halcyon(_dot_)com>
       _-\_<,       -     _\ /\_       15307 NE 202nd Street (206)489-0501
      (*)/ (*)     -     (*)^(*)     Woodinville, WA 98072
     ------------------------------
     More comfortable AND faster ... that's REAL technology!

====== start of ~/newdig.txt =====

NEWSDIGEST(1)            USER COMMANDS              NEWSDIGEST(1)

NAME
     newsdigest - a batch oriented newsreader

SYNOPSIS
     newsdigest [ -cdemv ] [ -t to ] newsgroup
     newsdigest [ -cdemv ] [ -t to ] newsgroup article ...
     newsdigest [ -cdemv ] [ -t to ] message-id ...
     newsdigest [ -acdemv ] [ -f filterfile ] [ -n newsfile  ]  [
     -t to ]

DESCRIPTION
     Newsdigest selects and collects news articles,  sorts  them,
     and  prepares them for mailing.  Articles can be selected by
     keying upon header fields.  Articles can be sorted into vir-
     tually  any order desired including subject order and thread
     order.  Newsdigest is designed to  be  run  periodically  by
     at(1)  or cron(1).  It can be used for (at least) three pur-
     poses:

     Forwarding
          Newsdigest can be configured to send news  by  mail  to
          users  on  remote  machines  that  do  not  have a full
          newsfeed.

     Archiving
          Individuals who are highly interested in certain  news-
          groups  may  wish  to  have  articles archived for them
          while they are on vacation or otherwise unable to  read
          news.

     Massive Selection
          Newsdigest can be given an  incredibly  complex  filter
          file  and  a huge list of newsgroups to scan.  It might
          then be run unattended at some time in  the  middle  of
          the  night  when resource use and response time are not
          issues.  Newsdigest has  been  designed  and  optimized
          with massive selection in mind.

     Newsdigest encapsulates messages into  an  internet-standard
     digest  that  can  be burst back into into their constituent
     messages for convenient reading by some mail readers such as
     rmail  and  MH.  Unfortunately, the majority of mail readers
     currently available do not yet provide this functionality.

     The created digest is normally written to  standard  output,
     but  if  the  -m option is present, and if the digest is not
     empty, it will be automatically mailed instead.

     News articles are obtained by making a network connection to
     a  NNTP  news  server.   The name of the host providing NNTP
     service is stored in a file maintained  by  the  person  who
     compiled  newsdigest. It is possible to override this choice
     by setting the NNTPSERVER environment variable.

     Articles that are copied into the digest have certain header
     lines  of limited use removed.  The header lines that remain
     are sorted into a  predefined  order.   Sorting  the  header
     lines  in  a predefined order permits rapid scanning of mes-
     sages.

  Newsgroup Subscription Information
     Current  newsgroup  subscription  information  is  kept   by
     default  in  $HOME/.newsdigest. This filename can be changed
     with the -n option.  This file has one entry per line.  Each
     entry  has  a  newsgroup  name,  a  separator  character, an
     optional modifier character, and the number of  the  highest
     read  article  in the newsgroup.  The separator character is
     either a semicolon or an explanation point.  If the  separa-
     tor  character  is  a  semicolon, then the newsgroup is con-
     sidered to be subscribed to and the unread  articles  in  it
     are gathered and filtered.  If the separator character is an
     explanation point no articles are retrieved  for  filtering.
     If the modifier character immediately after the semicolon or
     explanation point is an equals sign, a subject line  summary
     of the new news is placed in the digest's introductory text.

     If newsgroups are found to exist that are not listed in  the
     news subscription information file, they are inserted at the
     end.  The arrival of new  newsgroups  is  announced  in  the
     digest's  introductory  text.  Initially, new newsgroups are
     not subscribed to.  This behavior can be reversed by  giving
     the -a option.

     If the same newsgroup is listed  in  the  news  subscription
     more than once, the duplicates will be removed and a warning
     message will be issued.  If newsgroups that do not exist are
     listed  in news subscription file they will be removed and a
     warning message will be issued.

     When newsdigest runs under the direction of a news subscrip-
     tion  file,  a  new version is written out at completion and
     the old version is backed up with a .bak suffix.

     The news subscription file can be initialized by creating an
     empty  file  and then running newsdigest. This will fill the
     news subscription file with  all  available  newsgroups.   A
     text editor can be used to subscribe to newsgroups.

  Article Selection
     Article selection is accomplished with  an  awk(1)  program.
     The program is by default stored in $HOME/.newsfilter.  This
     filename can be changed with the -f option.   If  this  file
     doesn't  exist or isn't readable, all articles in subscribed
     groups are presented in the order that they  were  received.

     When  the  awk(1) utility is run to interpret the filterfile
     and a tab separated list of headers taken from unread  arti-
     cles  are  presented.   For  each  article  that  is  to  be
     selected, the program should output the message-id  and  two
     numeric sort keys followed by zero or more alphanumeric sort
     keys.  Upper case letters in all of the header lines  except
     for  the  Message-Id  line  are mapped to lower case letters
     before being presented to the selection  filter.   Any  tabs
     are  changed to spaces so that they will not be mistaken for
     field separators.  It is possible  for  some  fields  to  be
     empty.   The  references field isn't taken from the message.
     Rather, it is computed by newsdigest.  This guarantees  that
     it will be well-formed even when a message contains garbage.
     It is also sometimes possible for  newsdigest  to  determine
     that  a  message  has  more  references  than  it explicitly
     claims.

     Here is a sample selection  filter  file.   It  selects  any
     articles  from  Ray(_dot_)Moody(_at_)Cray(_dot_)Com from any subscribed 
news-
     group and presents them before any other news.   It  selects
     articles  from  rec.humor.funny  that  do not have "adminis-
     trivia" as a keyword and presents them  in  the  order  they
     were  received  by  the  system.   It  selects  articles  in
     rec.travel and presents them in thread  order.   It  selects
     articles  in rec.roller-coaster and presents them in subject
     order.

          BEGIN {FS = OFS = "\t"}

          #     $1 - message-id
          #     $2 - linenumber in $HOME/.newsdigest
          #     $3 - current article number
          #     $4 - newsgroup (just one)
          #     $5 - from
          #     $6 - newsgroups (a comma separated list)
          #     $7 - subject
          #     $8 - keywords
          #     $9 - organization
          #    $10 - references

          $5 ~ /ray(_dot_)moody(_at_)cray(_dot_)com/ {
               print $1, 0, 0
          }

          $4 == "rec.humor.funny"  && $8 !~ /administrivia/ {
               print $1, $2, $3
          }

          $4 == "rec.travel" {
               if ($10 == "") {
                    print $1, $2, 0, $1
               } else {
                    print $1, $2, 0, $10 " " $1
               }
          }

          $4 == "rec.roller-coaster" {
               if (substr($7, 1, 4) == "re: ") {
                    print $1, $2, 0, substr($7, 5), 1;
               } else {
                    print $1, $2, 0, $7
               }
          }

     On systems where gawk(1) is available, it will  be  used  to
     interpret  the awk(1) program.  On systems that have nawk(1)
     but not gawk(1), nawk(1) will be used.   These  interpreters
     have  more  advanced  features such as the ability to create
     user-defined functions and a  random  number  generator  for
     random selection of articles.

     It is possible to use awk(1)'s piping  facility  to  perform
     selection filtering with programs written in languages other
     than awk(1).

     Articles that appear in multiple subscribed newsgroups  will
     be  presented  for  filtering multiple times.  Articles that
     are selected by the filter more than once will only  be  put
     into the digest the first time they are selected.

     If newsdigest is given a group name and a  list  of  article
     numbers as an argument, it will retrieve the requested arti-
     cles.  This can be used to select articles of interest  from
     a subject line summary.

     If newsdigest is given a group name but no article  numbers,
     it will produce a subject summary.

     If newsdigest is  given  a  list  of  message-ids,  it  will
     retrieve  the requested articles.  Be aware that message-ids
     necessarily contain characters, namely  < and  > , that have
     special  meaning  for most shells and will need some form of
     quoting.  For example:

     $ newsdigest -c 
'<1992Jan9(_dot_)104845(_dot_)23964(_at_)walter(_dot_)cray(_dot_)com>'

OPTIONS
     -a   The -a option causes an automatic subscription  to  any
          new newsgroups that have arrived.

     -c   This option causes articles to be concatenated together
          without  any  mail  headers  or  digest  trailers.  The
          resulting digest will be less cluttered, but  it  can't
          be directly mailed or burst by a mail reader.

     -d   The -d option is intended only for debugging.  No  con-
          nection is made to the NNTP server.  Instead, NNTP com-
          mands are issued to standard error and replies are read
          from standard input.

     -e   This option alters the article encapsulation  algorithm
          to  perform  better  with  the  emacs-based mail reader
          rmail.  The rmail undigistifier is substandard in  that
          it doesn't recognize RFC 934 character stuffing.

     -f filterfile
          Use the awk program stored  in  filterfile  to  perform
          article     selection     processing     instead     of
          $HOME/.newsfilter.

     -n newsfile
          Read and write  subscription  information  to  newsfile
          instead of $HOME/.newsdigest.

     -t to
          Specifies a mail address for the  "To:"  line  for  the
          output digest.

     -v   Be verbose.  All NNTP commands as well as  the  initial
          line of the response are logged to standard error.

FILES
     $HOME/.newsdigest
          list of available newsgroups, subscription information,
          and already presented article numbers.

     $HOME/.newsfilter
          awk(1) program to select or kill articles.

AUTHOR
     Ray Moody <Ray(_dot_)Moody(_at_)Cray(_dot_)Com>
     Cray Research Inc. field support
     Cincinnati Ohio

SEE ALSO
     at(1), crontab(1), awk(1),

     Marshall T. Rose and Einar A. Stefferud,  Proposed  Standard
     for Message Encapsulation, RFC 934.

     Brian Kantor and Phil Lapsley, Network News Transfer  Proto-
     col, RFC 977.

     David Brin, Earth, Bantam Books, 1990.

DIAGNOSTICS
     Exit status is 0 if all goes well or  something  taken  from
     <sysexits.h> in the event of failure.

BUGS
     People who want to use this program shouldn't be required to
     learn awk(1) and either at(1) or crontab(1).

     Subject line summaries can't be  produced  without  an  NNTP
     server  that  has the XHDR extension.  Fortunately, the XHDR
     extension is widely available.

======= end of ~/newdig.txt =====

<Prev in Thread] Current Thread [Next in Thread>