|
Re: Mailchain optimisation [Long and possibly OT]
2009-05-30 12:30:55
At 13:22 2009-05-30 +0100, Arthur Dent wrote:
This is a home server. I am not a sysadmin just an interested hobbyist.
You're not a sysadmin by profession - you're still a sysadmin if it's your
machine and you're responsible for the administration of it.
fetchmail->procmail(->clamav->spamassassin)->dovecot
I use mbox format (not maildir).
Ah, the fetchmail bit wasn't mentioned previously.
If the host on a static (fixed, unchanging) IP, or dynamic? If it's
static, there's a lot you can to to improve things, esp for the vanity
domain - your other domains would still be fetchmailed.
I have mail accounts with at least 4 different ISPs at least one of
which is a catch-all address. One is a vanity domain address and is of
the form whateverIwant(_at_)myvanitydomain(_dot_)demon(_dot_)co(_dot_)uk (the
".demon.co.uk"
bit is real).
Right, so three accounts are just mailboxes at those respective ISPs, where
you have nothing to do with the domain, while the last one you _might_ be
able to have the MX record updated to point to your host, and a secondary
to the mailhost of that ISP.
I don't use demon, so I've no idea if they permit you to have an MX in the
chain. Basically though, if your home server is on a static IP, you set it
up to handle mail for vanitydomain.demon.co.uk, and set up the individual
virtual users, and perhaps initially, a catch-all (until you clean things
up), then request demon to make DNS changes to set your home server up as
the primary MX for that host, and their mail server to be the backup
MX. When a message comes along for you, it'll try to deliver to your home
machine first, and if it's not accessible for some reason, it'll go to the
backup MX at demon, and periodically, they'll try to send it along to the
primary (your home server).
If your home box isn't on a static IP, then you need to look to use a
dynamic DNS service, and set up a cron job to update your IP information
with that service (I don't use any of them myself, but this is how it works
more or less). Then, when you know the dynamic DNS stuff is working, go do
all the stuff you'd be doing for the MX change above, but specifying your
dynamic DNS hostname (you can use the dynamic dns service domain for the
hostname - no need to try to get it registering as a host within your own
domain, which really would just be a CNAME to the dynamic DNS host if you
had your own domain). If your host goes offline and back online (say, if
you were on dialup), you can use fetchmail to "tickle" the backup MX to
process its queue, rather than waiting for the backup MX to "get around to
it" -- ETRN is the SMTP command, and you should find that in the fetchmail
docs, if you need it.
I used to use this extensively:
me@@myvanitydomain.demon.co.uk
mybusinessname(_at_)myvanitydomain(_dot_)demon(_dot_)co(_dot_)uk
wife(_at_)myvanitydomain(_dot_)demon(_dot_)co(_dot_)uk
son(_at_)myvanitydomain(_dot_)demon(_dot_)co(_dot_)uk
and also for creating a separate identity for each website or
mailinglist I registered with - the idea being that I could keep track
of any misuse of my email address.
I have since come to realise that this is of little benefit and I have
moved away from this
Indeed - if you cannot expressly disable those individual email addresses,
what you're doing is effectively increasing the number of targets spammers
have. I generally use a disposable hotmail account for registering with
sites where I'll rarely use them. You should be able to use PLUSSED addresses:
misc(_dot_)lists+procmail(_at_)blueyonder(_dot_)co(_dot_)uk
at least on your own server (where you're running sendmail). Some of the
other services you're using may or may not support plussing. On your own
server, you can expressly disable different plussings if need be -
otherwise, plussing acts as a wildcard (because the base email address is
legitimate - the plussing is a local extension).
Mail is delivered into /var/spool/mail/username (currently 4 - root,
mark, wife, son). I am assuming this is the default Fedora setup as I
don't remember changing anything, but I confess I don't really
understand it.
I run fetchmail as user "mark" (I believe that running fetchmail as root
is deprecated). This collects POP mail from each of the 4 ISPs.
Fetchmail hands off to procmail. In my /etc/procmailrc I have the
following settings:
SHELL=/bin/bash
PMDIR=/home/mark/Procmail
This is not a standard procmail variable, so if you're using it, you're
expressly prefixing something with it: INCLUDERC=$PMDIR/somefile.rc
LOGFILE=/var/log/procmail/pmlog
hock a:
VERBOSE=ON
in there some time, and watch the detail. Without it, you're just seeing
delivery summaries.
1) Remove any well tested regular spam patterns and consign them to the
/dev/null pit of doom
As a rule, I _save_ spam. I've already taken the network hit if I'm
processing it post-SMTP (if you're in procmail, you are), and storage is
cheap enough these days. Thus, if by chance, a message is miscategorized,
it's not been lost. It also helps when you make changes to filters to have
a big steamy pile of spam to throw at a test filter (see "sandbox" in the
URL in my .sigline) to see how it'll react.
2) Extract any trusted mailing lists (such as this one) and deliver them
directly into organised directories
There's a listname identification rcfile that's been discussed several
times on this list (also available via the URL in my .sig) - this makes it
easy to identify lists, and you can then deliver to auto-named mailboxes,
or match against a list of "good" lists. For example:
INCLUDERC=listnameid.rc
:0
* LISTNAME ?? (procmail|php-general|v12-engine|mysql|bugtraq)
$LISTNAME
The LISTNAME variable remains available for subsequent evaluation in other
rules (I use it to bump up my spammishness allowance for certain lists more
prone to spammy discussions).
3) Filter against a list of my known addresses (see above) anything
addressed to an unknown name goes into a "possible spam" folder for
review. This is still required because someone may mistype one of the
real names or it may be from something I registered in the past but have
not included in my list yet.
Which is where you want to consider NOT using a "catch-all" wildcard
address on your demon account - if possible, list out all the addresses you
use (some grepping of your saved email should net you a scrappy list), then
set them up as the only valid addresses, and cease using the
wildcard. People who mistype an address should get a BOUNCE.
I sent an invoice to someone at an address which was miscommunicated, and
received no bounce. Some time later, I called and asked about the status,
and found that they'd not received it - in checking the newly conveyed
address, there was one less (or one more?) 'r' in a name - but because
there was an unchecked catch-all, they hadn't retrieved the message, and
because the catch-all was present, my own message wasn't
bounced. Catch-alls are evil not just because they make an easy target for
spammers using dictionary methods.
Anything returning unmolested from that lot then gets filtered for
wife/son and delivered to /var/spool/mail/username. This is the bit
which Sean feels should be done by sendmail (and I agree).
'cepting that you're fetchmailing, which wasn't originally disclosed. If
fetchmail is pulling these messages down from a single mailbox, via POP for
instance, you don't have envelope data - it's already been "delivered" when
your upline ISP received it and put it into the mailbox you're now
retrieving it from. Once that happened, the envelope data went "kerpoof".
Then (still in /etc/procmailrc) I have another INCLUDERC that sorts my
mail into various mail folders that I have organised (I know that this
should be done by ~/.procmailrc but I can't get that to work - see
below).
Anything left drops through into /var/spool/mail/mark
This is because of $DEFAULT, or because you're expressly doing it in the
global rc?
Problem 1
=========
What triggers $HOME/.procmailrc ?
A message which isn't already delivered by the etc/procmailrc. If your
global rc delivers the message, procmail will stop processing, never
loading up your personal one. You might also have permissions issues on
the rc file that makes procmail decide not to open it - if so, it should
get logged.
I do have a /home/mark/.procmailrc but it does not seem to get used (I
have tried) and hence I do my personal mail folder organisation using an
INCLUDERC off of /etc/procmailrc.
Is this because the mail goes into /var/spool/mail/username?
Because the global rc is _delivering_ it there? Yea, that'd do it.
Problem 2
=========
How do I rope sendmail in to help me here (apologies for OT)?
I have tried the following experiment:
I created the following entry in /etc/mail/virtusertable
joebloggs(_at_)myvanitydomain(_dot_)demon(_dot_)co(_dot_)uk son
Is your host configured such that myvanitydomain.demon.co.uk is in it's
list of local-host-names?
(and ran makemap hash /etc/mail/virtusertable.db <
/etc/mail/virtusertable) and restarted sendmail for good measure.
I also allowed the joebloggs entry through my filter.
I changed the MAILDIR directive to be MAILDIR=$HOME (and also tried
MAILDIR=$HOME/Msgs)
MAILDIR is in effect, a CD command. INCLUDERCs, and file deliveries which
are not prefixed with a directory (or which are prefixed with a relative
dir) will occur within the directory specified by MAILDIR.
I then sent myself a mail to that address. As it did not touch any of
the aforementioned filters it dropped straight through but was delivered
into MY Inbox - not "son" as I would have expected / hoped. (On a couple
of tests was delivered as msg.VSlK or somesuch - but still in MY /home).
Well, the message isn't arriving at your host via SMTP - fetchmail is POP
or IMAP'ing the message and invoking procmail with it - so your SMTP config
isn't being used.
Why? Is it perhaps because fetchmail is called by user "mark"?
Well, there can be issues with that, when you're trying to deliver to the
other users, but per above, it's because sendmail isn't part of the picture.
Any tests I can try to see if sendmail is actually registering the
virtusertable entry?
sendmail -bv address
---
Sean B. Straw / Professional Software Engineering
Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
Please DO NOT carbon me on list replies. I'll get my copy from the list.
____________________________________________________________
procmail mailing list Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail
|
|