Re: Certified Mail Delivery agent

At 03:06 2001-12-12 -0500, Paul Chvostek did say:
> Sean, I'm relieved to see that you jump on *everyone*, and not just me.
> ;-)
Hmm, I quicky reviewed my recent followups to your posts, and I spot only 
two threads in the past few months - both of which were answered without 
even coming close to feeding you to the lions.
> The impression I got was that the program itself "certifies" the mail by
> making a delivery confirmation request to the sender.
Certifies the sender then.  Sort of.

> *wrote* some software does not seem to be grounds to kick him in the
> teeth.  Lighten up a little!
I'm not kicking anyone in the teeth, and certainly not with steel-toed 
boots.  Nor have I ever done the same to you.
> > Replies according to some special text, or just replies?  What if the 
> reply
> Good point.  What would you suggest?  Would asking people emailing you to
> put the word "accept" on a line by itself near the top of the message
> confuse too many people?
I think certainly people who end up receiving the "explain yourself" 
notification when they didn't directly address the recipient (and in fact, 
may not even know them) will be rather confused by the request.  Sending a 
message to someone saying "we got your message, now, as if you were 
subscribing to a mailing list, we expect you to reply to this message with 
some unique text (clicking "reply" simply will not do because bounce and 
autoreplies will render the system useless), before we'll ever bother to 
read what you send" is going to put people off.  Frankly, it screams "I'm 
too busy to bother reading your email unless you take extra steps for me," 
and invites the sender to not bother sending further messages (provided 
this autoresponse isn't generated via mailing list traffic where they can't 
control that you're receiving the messages).
FTR, in an ironic twist, such autoreplies, if sent out in response to spew 
will only CONFIRM your address to spammers if they sent their message from 
a valid account.  You're certainly not doing yourself any favours by doing 
that.
Not sending them in response to messages which have "remove" or somesuch 
near the bottom will of course cause some spew to pass unhindered when they 
have removal instructions.  Or am I mistaken about what happens with 
messages which are not sent these autoreponses - do they pass unhindered, 
or are they subject to manual intervention on the part of the recipient?
What happens when the sender says "oh, screw that" they first time they get 
asked for a confirming reply, puts it out of their mind, and six months 
later, sees a message from you in a public forum (such as this one), takes 
the time to construct a meaningful reply and sends it off to you?  If the 
agent is sensible, it'd go "oh, I already asked this bloke for a 
confirmation, and they didn't give me one, so I'm not going to continue to 
pester them" - uh-oh, you'll never get the message and they won't be told 
as much.  But if the script ALWAYS sends the response when the person isn't 
in the acceptlist, then you risk the ire of people who may receive them on 
a mailing list in response to every post.  A midpoint would be to have the 
notification function use a cache which says "I asked this bloke for a 
confirmation, but that was months ago - let me ask again".  Still, you run 
good odds that you'll just annoy the sender when you remind them that 
you're still not interesteed in hearing what they've got to say unless they 
take some extra steps you require of them.
If the message arrives without any indication as to what it is in autoreply 
to, that'll be even more puzzling/annoying (like vacation replies generated 
from messages received by a mailing list recipient, which have been 
delivered to the author of the message, but often toally out of sync with 
the original message transmission -- with no indication of an original 
subject and no body text - you're left wondering not only who this idiot is 
that is mailing you, but also from what mailing list they're receiving your 
mail).
> What about ignoring any apparent confirmation that contains the word 
> "automated" in un-quoted text?  Or perhaps ignoring apparent confirmations 
> that contain *any* unquoted text at all (i.e. 2 or more lines before a 
> '^-- $' that match '^[a-zA-Z0-9]' ?
May as well pipe the message through an HTML stripper in case they reply in 
html text (say, with the bane of mailing lists, AOL6 which apparently lacks 
a "plain text" option).  Then there are those annoying corporate mailers 
that stuff things into the outbound message like it or not.
> > Q: how well does this work for _mailing lists_ -- does it send the request
> I don't know what John's software does.  But what would you consider
> safe behaviour for this?
Safe behaviour would start with applying the same logic that one should to 
regular autoreplies: common list identifiers (as you described, but also 
including a sender which matches "(.*-admin|owner-.*)@", as well as other 
characteristics) would be indicators of message sources which should not be 
autoreplied to - any "certified" delivery mechanism would be better off 
collecting those addresses and presenting them to the intended recipient as 
"hey, I've received mail from the following address which appears to be a 
list but is not currently in your acceptlist - reply to this message if 
you'd like me to add it, or reply with 'xxx' if you want me to do a 
one-time forward of this specific message."  Push the administration on the 
user who elects to use it, since they're the ones interested in filtering 
their mailbox.
THEN, any message in which the recipient isn't clear copied (TO/CC) should 
be suspect - these are the messages you most likely shouldn't autoreply to 
either (mailing lists somehow not caught with the normal mailing list 
conditions) and, rather ironically - the most likely source of spew (BCC'd 
spam).  So, to reliably catch those without being obnoxious to mailing list 
participants, you still need to run your messages through spam filters.
IME, limited acceptlists are effective for allowing certain senders 
unfettered access to your inbox, while a judicious application of spam and 
twit filtering catches the bulk of the spew - generally without subjecting 
valid email to unnecessary risk.
> Obviously, any advice that you can provide to make this tool more
> effective will be appreciated.  If you *really* want to know how
> John's software works, just read the source.
I _really_ don't plan on using anything which sends messages to people 
asking them to defend their action in delivering a message to me: such a 
process only annoys valid senders, since the spammers (presumably the 
people you expect will not reply to a message because their sending address 
isn't valid) won't actually receive the certifification request message.
This post may seem like harsh criticisms, but the intent is wholly 
different - I ask the questions in an effort to raise awareness of certain 
issues which will arise from common flaws in implementations.  One doesn't 
test an armoured tank by throwing water balloons at it, and you don't go 
into battle without tested gear.
> > >Although technically feasible, my tests show that it is annoying in
> > >practice, and few senders reply to the certification request.
> >
> > Which means the user of the script will be faces with a lot of trashbinned
> > mail they must manually dredge through to check out because the senders,
> > while at the same time annoying others through its existance.
>
> Yup, that's one of the down sides to implementing filters.
Uh, my comment was *specifically* aimed at the author's own admission 
quoted above -- for now, ignore the "annoying in practice" part (which I 
entirely agree with) and focus on "few senders reply to the certification 
request" part -- that means by implementing such a filter, sure, you get 
rid of a large amount of spew -- but along with it, you get rid of a large 
amount of perfectly valid email because many of those authors don't want to 
bother with jumping through hoops so you'll accept their email.
Want a acceptlist?  Take your existing saved mail, and ensure that all the 
spew is removed from it, then pump it through a script that extracts sender 
information and adds that to your database.  Then, each message which comes 
through which doesn't get identified by the acceptlist can be detained 
while the recipient is notified of it's arrival.  I think a modified 
vacation ruleset would do most of the work (good vacation rulesets take the 
senders address and add them to a datafile so that you don't send multiple 
announcements -- just as you woulnd't want multiple notices for the same 
"new sender".
> I'm almost finished writing something similar to John's CMD
> which includes something that'll let users forward saved messages to
> a magic address which will automatically add sender email addresses
> to the user's whitelist.
IMO, this is a much less obtrusive approach - the recipient is validating 
the message as good, and the sender (who for all we know may be replying to 
a list message with useful information in response to YOUR query), doesn't 
have to deal with authenticating themselves as a valid person to suit the 
whims of the recipient (after all the sender already took the time to 
compose a reply to the original query).
I must now wonder about web bots which email out confirmations "here is 
your new password to our service", listserv welcome messages, or funky 
bounces (some cheezeware MTAs send bounces FROM the address which is no 
longer valid - so an autoreply to that address will result in a subsequent 
bounce -- and YOU won't know your original message to this individual, 
perhaps in response to a list message someplace, has bounced).  If such a 
ruleset receives these types of messages, and the addresses aren't in the 
acceptlist, so it generates a demand letter, is the bot all that likely to 
respond favourably?  Seems like you'd be digging through the archive file 
examining messages.
I think autoreplies are iffy enough -- one shouldn't rely on such 
mechanisms for confirming sender validity.
> When I get the code for this finished, should I post it to the list,
> or just keep quiet about it in order to avoid invoking the cholor of
> Sean B Straw?
Do you insist on making this out to be a personal attack?

---
 Sean B. Straw / Professional Software Engineering

 Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
 Please DO NOT carbon me on list replies.  I'll get my copy from the list.

_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail