Re: processing emails that bounce.
2002-05-21 14:31:17
At 13:07 2002-05-21 -0700, Teddy Campbell did say:
What would be the best way to catch emails that bounce? Heres
something similar to my situation.
Why not just describe _your_ situation?
I have a mailing list of people who want to get a news letter. Some of the
people no longer have their email addresses so the email bounces.
This is a common issue on any mailing list. However, the reasons for the
bounces go far beyond the accounts merely being closed:
* user has mail filtering (I hate it when AOL'ers decide this is how
you unsubscribe - by "refusing email from")
* mailbox is full (you might not want to remove someone from your
mailing list just because you get one of these -- a bunch
however, and they're outta here)
* final delivery cannot be made - intermediate server is bouncing you
notification of this
* DNS and MX configuration errors ("we do not relay" is one)
* connectivity issues (remote host disconnected from the internet).
* local config problems (incl. disk full) at the recipient host.
plus a variety of others.
Often, the actual cause isn't even given in the bounce - you're just told
the message couldn't be delivered.
One problem you'll find out in a hurry if you have much email traffic is
that there are a LOT of truely abysmal MTAs out there (many of which are
cheezy solutions for MS Windows based systems - I'm not knocking MS
Windows, just the plethora of crap people choose to use on it). Some of
these MTAs will bounce a message to the FROM: address rather than to the
sender. There's even a few that address bounces to the original TO: address!
Ensuring that your list rejects posts from non-subscribers is a good first
step in blocking such idiots. Adding loop checking (to ensure that a
message which has already passed through your list isn't being sent back
into it, say by some braindead wannabe MTA that ignores the envelope data
and tries to deliver the message to the original cleartexed recipients --
usually your list) is another thing which should be done -- when you
experience your first mailing list (whether you're the admin of a list or
not) loop because some yo-yo's mail server is resubmitting messages, you'll
have some respect for the grief -- but when your the sysadm of the server
with a LOT of subscribers, the grief is really compounded.
Now, even if you were to somehow deal with the above problem MTAs, you're
still stuck with the many which don't bounce an actual address identifier
for the user which they're refusing to deliver for. Oversimplified "your
message could not be delivered" or "could not deliver to user joebob, bob"
(without address). Add to these messages which do identify a user address,
but it's a rewritten address: bob(_dot_)joebob(_at_)corp(_dot_)somecompany(_dot_)tld instead of
"bob(_at_)somecompany(_dot_)tld" who subscribed to your list. Don't forget delivery
failures on the last leg of a FORWARDED subscriber: user subscribes as
user(_at_)onebox(_dot_)com, which forwards to differentuser(_at_)yahoo(_dot_)com(_dot_) When yahoo
bounces, it isn't going to provide you with the email address that is
actually subbed to your list.
Some of these, you might discern the original recipient from an included
set of headers from the message which was being bounced. Or not (esp if
there were multiple recipients at the domain in question).
Some DSNs include multiple email addresses - with DIFFERENT failure reasons.
I could go on and on here...
What some email list managers do is set the sender address to a unique hash
representing the subscribed address - when a message bounces (properly, but
not necessarily with useful info in the body), the address it was sent to
can be used to automatically determine the actual subscriber for that
bounce. Of course, this means that EVERY MESSAGE through your list will be
delivered as a UNIQUE message out to EACH RECIPIENT. Normally, if you
delivered a list message to 20,000 subscribers, 500 of which happened to
have hotmail accounts, there'd be ONE connection to the hotmail MX to
attempt delivery of the message to all of the subscribers there. That
makes the email exchange efficient - one copy of the message and a
buggerin' large list of envelope recipients. To uniquely hash each message
means that you'd have 500 message transactions with hotmail instead. Icko,
esp on large lists or those with lots of traffic.
A workaround (if the list has enough traffic to justify it), is to handle
regular list messages normally (in batch), but to send out a monthly
subscription probe which is handled with a per-user hash. This
subscription probe could be disguised as a FAQ or other informational
message, to make it useful to the recipients.
What would be the best recipe to have procmail make a list of the email
addresses that have bounced?
First would be to ensure that the list sender is an address separate from
the list. The accepted standard is an owner-listname alias -- you could
direct this to your individual address, or through an invoked procmail
script which in turn delivers to you after pre-processing the messages.
A few months ago, I wrote a collection of procmail scripts to work in
conjunction with several majordomo lists I'm involved with
administering. These scripts handle messages being sent into the lists
(attachments, HTML mail, idiots reposting complete digests in reply, etc),
as well as the owner aliases, to which bounces issued by the majordomo
processor - non-subscribers, mismatching subscription auths, keyword
bounces, etc. are sent, as well as the majority of delivery bounces.
Owing to the wide variety of bounces and the issues outlined above (and
many more), the listowner filtering doesn't automatically take action on
problem addresses -- it does however afford a central place to put filters
to eliminat certain nuisance bounces - say, when @home closed up shop, and
all those addresses started bouncing -- after they'd been in the queue for
five days, which resulted in five day lags on the bounces AFTER you
manually unsubbed all the users. Centrally filtering them allowed those
specific bounces to be weeded out and tossed rather than driving the
listadmin insane. Likewise for some ISPs which insist on sending DSNs for
messages every four hours for a week because their customer's email is
handled over what must be a dialup...
These are handled on a case-by basis.
Something you should look at doing is separating out the DSNs which are
emitted by your local server, versus those which are emailed back to
you. Note the different types of failures reported.
If you come up with THE solution, please do share it -- the world is
waiting for it.
---
Sean B. Straw / Professional Software Engineering
Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
Please DO NOT carbon me on list replies. I'll get my copy from the list.
_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail
|
|