procmail
[Top] [All Lists]

Tips for Newbies, from a Newbie

2002-03-11 11:41:57
This is long, but it would have made my procmail life 
much easier had I encountered this info, in one place, 
early on. Instead, I learned by the seat of my pants, so 
I pass it on with the  sincere hope it will assist 
others. It describes Windows-tools proving VERY helpful 
for recipe development, as well as a few things I've 
learned along the way.

Short version: a few months ago, my procmail IQ 
began/ended with the ability to spell the word. I 
treated all but the most simplistic recipes in the same 
manner I'd greet an approaching vampire. But, my primary 
website-related mailbox continued to receive its daily 
average of 70+ spams, so I HAD to do something. Action 
blossomed, with a vengeance, the day I saw an "Eat my p--
--" subject line  in my box. In essence, I went nuts! 
[short drive, but still...]

Now, I have a fairly elaborate procmail scheme running 
on my hosted Linux account. I've seen no spam in 3 
months;false positives have been conquered; some spam is 
forwarded to ftc.gov; some is deleted outright; some is 
filed as evidence for a tentative lawsuit. Further, 
copies of  my Blackberry incoming/outgoing 
correspondence is now completely organized upon receipt; 
and, I have finally been able to confine use of Outlook 
to personal info management and syncing, exclusively. 
Happy Campers R Us!!

Explanation: After stumbling about, I ultimately sat 
down and outlined what I needed to do, to restore sanity 
to all things email. Understand: the process I chose 
follows my central belief: you either pay on the front-
end by doing your homework, or you pay on the continuum 
via mistakes with elusive remedies.  Since I suffer from 
LTL disability [LongTerm Laziness], I opted to do my 
homework up-front:

[1]  create my own library of procmail info
         DL **all** pertinent man pages [ /info basics]
         procmail FAQs [ /info basics]
         mini-tutorials [ /info files]
         Timo's/Jari's/Nancy's pages [ /info files]
         4/2000-current procmail list archives
                            [ /info files]
         all branching off my working directory
[2] get intelligent Win tools [both free, no ads/spyware]
        CrimsonEditor(.com)
            kills trailing spaces on save
            saves as Unix
               (comparing bytes **quickly** confirms
                 the latest revision WAS u/l'd)
            color highlights syntax
               (major whoopie!)
            view hidden tab character
               (thankya Lawd!)
            set working directory
            enable auto-indent
            search through directory
            [more below]
        NoteTab Light [webattack.com]
            convert html pages to nicely
            formatted plain text via drag
            and drop (enhancing searchability)
[3] develop recipes, including a LOG= line
        for each and every one; keep on
        verbose logging and study file
        daily, including 30-day period AFTER
        consistent confirmation of no errors
        for a one week period
[4] massage rc files so workhorse recipes
        appear early in the filtering sequence

Crimson comes with, among others, Perl syntax 
highlighting built-in. There are 3 related perl.* files 
within Crimson's directory tree. I opened each, tweaked 
them, and saved under rc.* names. I created a new file 
group, calling it Procmail Recipes, and added an 
associated *.rc group of files. I then tweaked the color 
coding, so that |\, |, :, !, { }, [ ], ( ) and a few 
other gotcha's always appear in red--- thus decreasing 
obvious goofball mistakes I tend to make while working 
on this stuff during wee-hours. Comments are brown, & 
mailbox destinations are blue. I went back and similarly 
created a syntax group of files to expedite log review: 
errors [skipped, overflow, exceeded, etc.] appear in 
red, "notified comsat" in blue, etc. Result: when I open 
RC files, the color coding makes errors immediately 
apparent because they just about jump off the page. 
Supremely helpful for long alternations/OR-condition 
statements!!

Crimson's search facility is super-handy. The hitlist 
appears in a bottom window, with surrounding text re: 
your search term. Doubleclicking a line takes you to the 
precise associated text in the file concerned. Caveat: 
during  sessions where multiple huge files are 
opened/closed,  Crimson sometimes eventually ignores my 
search-directory command. Shutting down the program and 
restarting it cures this. Irritating, but a tiny price 
to pay considering all the goodies it offers, esp. since 
the working directory is always visible in the left 
window.

As a result of setting things up in this manner, when 
errors were encountered in the verbose log file, I knew 
immediately which recipe in what file sparked it-- just 
by looking at the LOG= line (yes, I numbered each recipe 
per file). I could then search through my stash to find 
an answer if it wasn't an obvious "duuuhhh" [missing 
top :0 line, missing * at start of condition line, 
etc.]. With one exception, each and every question I had 
was quickly answered. One time, I didn't even have 
enough savvy to devise a keyword, so I just ran a search 
for |\     After jumping through assorted files, I saw 
what I needed to see, and comprehend, to cure the 
problem. Took less than 5 minutes, a chore which, 
without Crimson, previously took the better part of an 
evening!

The troublesome question? My website host 
employs .redirect files, rather than .forward. Some of 
my recipes were being ignored, yet there was no 
associated error in the log file. I ran google searches, 
and finally discovered that the .redirect file is 
processed before the .procmailrc file, as well as the 
distinction between .redirect files [virtual 
users/aliases] and .forward files [actual users with 
assigned boxes]. I removed the problem lines from 
the .redirect file, then substituted  related recipes, 
and the problem was resolved.

In retrospect,  in order of helpfulness and productivity 
enhancement:
   - the personal library created
   - CrimsonEditor
   - LOG= lines per recipe
   - verbose log review
   - reshuffling rc files order

Finally, FWIW, here's my OK-this-is-it-for-real-now RC 
order. Because my brain cells like to go on holiday 
without permission, I protected myself by going back and 
adding numbers to the file names, so I will always keep 
the workhorse order intact:
The "a" group = absolutely essential---
  1a_alwaysUse.rc
         grab headers per file
         all admin messages [cron, bounce, etc]
         daily jerks
            predictable idiots with predictable header 
snippets
  2a_ListsNews.rc
         newsletters & mailing lists to which I subscribe
  3a_Trancell.rc
         [explained below]
The "b" group is also important:
  4b_Virus.rc
        poison files, script language=
        headers captured so I can warn accordingly, if 
        desired
  5b_GoofyStuff.rc
        friends/family who insist on forwarding jokes or 
        funny things, the latest hoax because they don't 
        know better ["pass this on to all your 
        friends"], the 7+ recipient rule, segregate 
        messages bigger than xxkb, and a few others
  6b_OKcrew.rc
       after pre-screening for viruses and goofy stuff,
       these folks should always get through to my 
       Blackberry,with copies filed per my instructions
  7b_NoAlias.rc
       if it ain't addressed to me, it ain't for me 
       to see
       [kills about 80% of general spam, 95% of porn]
The "c" group kills what's left of spam messages.
  8c_Delete.rc
       X-tags, Message IDs, forged "freebie" addresses,
       and other standard recipes targetting obvious 
       spam. (in compliance with, click here|on|below, 
       meta...FrontPage, hello|dear friend|customer, 
       disclaimer, and other variants  of sho' nuff spam
  9c_ScreenDPG.rc
      = my primary spam-infested mailbox, which some 
      potential clients still access.
 10c_LastCall.rc
      grab everything else and file in 
      $MAILDIR/nofilters;
      rationale: I don't want anything in my $ORGMAIL 
      box except what I've dictated; this way, I can see 
      what the filters missed. For the same reason, I 
      set $DEFAULT to $MAILDIR/oops. Note: The date/time 
      stamps of Unix directories change each time 
      they're accessed by procmail. So, a glance at 
      that /oops directory tells me if anything was 
      dumped there since I last checked, 
      eliminating the need to look inside the directory 
      itself. 
As a direct result of this scheme, I was able to retire 
the following files, which were pain-in-the-skleeboop 
maintenance heavy anyway:
   -spambody.RC
   -spamFrom.rc
   -spamISPs.rc
   -spamProxies.rc
   -spamPorn.rc
   -spamSubject.rc
   -spamTo.rc
During recipe development, I adhered to 3 rules, with NO 
exceptions: include the Log= line per recipe; add an 
anti-loop to EVERY forwarded message, no matter where 
it's headed; document/comment purpose of each recipe.

Re: the Trancell.rc file, which will probably be of 
interest only to Blackberry users---- About 99.9% of my 
email is conducted with the Blackberry. And, via the 
free Trancell(.com) add-on service, I do roughly 90% of 
my google-based research and web surfing [web page by 
email retrieval] via the BB. My wireless provider, 
Earthlink, allows me to auto-forward a copy of  outgoing 
BB mail to my server. The problem is that I don't need 
copies of everything [i.e. 404 errors, daily news 
blurbs, get this page], yet forwarding is an all or 
nothing proposition. The trancell.rc file thus involves 
about a dozen recipes, placing google queries in one 
file, resulting hitlists in a directory [for 
individualized messages], correspondence in separate 
files [family, friends, clients, associates], requested 
pages in its own area, etc. Because they are plain text, 
I can use a [free] Boolean text-search program [see 
webattack.com] to find what I need when later needed, in 
a far more intelligent and expedient manner than if I 
were stuck with Outlook. Hence, I prefer to use FTP to 
transfer these files to my notebook for document 
retention purposes.

Bottom-line: I had been spending an average of 5-10 
minutes daily weeding out spam, despite Outlook's 
alleged filtering feature. Figure 8 minutes, times 7 
days, for close to an hour per week, 52 hours per year. 
I was therefore willing to devote 40 hours, up front, to 
fashioning a solution. Turns out, I came well under 
half  that self-imposed max, mostly over a 1 month 
period of concentrated focus via free time. That 
included diligently reviewing the procmail log every 
day, which eventually took less than 5 minutes per 
review, thanks to Crimson's one-click "find next" 
feature ["notified comsat"-- shows surrounding  lines 
with subject, plus my LOG=line and disposal].

Philip [one el ;-) ], Sean, Martin and others: I send 
THANKS!!!! in increments twice the number of bucks Bill 
Gates has in his account. I can't count that high, but 
you get my drift.


pam
--
Law Offices of D. Pamela Gaines
[NJ-PA-CT]
http://dpg-law.com
http://law4pda.org
The world is a circle; give back!
_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail

<Prev in Thread] Current Thread [Next in Thread>
  • Tips for Newbies, from a Newbie, webfiddler <=