Re: Split the folder and spread to the users

At 22:14 2002-05-05 +0300, Udi Mottelo did say:

>         Good people, I very need your help!   I made a mistake and
>         redirected innocent messages into the folder "vinfected".
Mentioning that you've done this from within a sendmail-invoked procmail 
rule, the /etc/procmailrc, or a personal .procmailrc which shares one 
physical account with multiple people.
>         The folder now has about 600 messages and I have more then
>         1000 users.  Is anybody has a good way to spread the messages
>         to the original destinations?
Repeat after me:

        PROCMAIL IS NOT AN MTA.

If these messages lack envelope information (and by the way, the MTA 
doesn't pass this information to the LDA), procmail can only guess based on 
headers.  This is part of the reason that some people configure their MTA 
to add an X-Envelope-To: header, which itself isn't a perfect solution, but 
does alleviate the problem.
You should become acquainted with your maillog - you could write a Perl 
script to pair up the topmost SMTP transaction id from each message with 
the SMTP id in your maillog, and extract the list of recipients from that, 
then invoke procmail (or the MTA even) to re-deliver the message as per the 
original envelope recipients.  No, I don't have a script to do this, nor do 
I have the time to dedicate to it - however, after you've written and 
debugged it based on my suggestion, perhaps you'll share it with the 
procmail community.  Or, I _might_ get around to writing such a script in 
six months (which does you no good today).
So many ideas, so little time.

To take my reciept of your message to this list as an example, the 
following appears in the locally delivered headers of the message:
> Received: from r220-1.rz.RWTH-Aachen.DE (r220-1.rz.RWTH-Aachen.DE 
> [134.130.3.31])
>         by trei.professional.org (8.10.2/8.11.1) with ESMTP id g45JDOS02538
>         for <PSE-L(_at_)mail(_dot_)professional(_dot_)org>; Sun, 5 May 2002 
> 12:13:26 -0700
> X-Envelope-From: procmail-admin(_at_)Lists(_dot_)RWTH-Aachen(_dot_)DE
> X-Envelope-To: <PSE-L(_at_)mail(_dot_)professional(_dot_)org>
Remember: the X-Envelope headers are added by my MTA config.

If I take the SMTP ID and search my maillog, I get the following hits:

> May  5 12:13:27 trei sendmail[2538]: g45JDOS02538: 
> from=<procmail-admin(_at_)Lists(_dot_)RWTH-Aachen(_dot_)DE>, size=4405, class=-60, 
> nrcpts=1, 
> msgid=<Pine(_dot_)SGI(_dot_)4(_dot_)21(_dot_)0205052127190(_dot_)747568-100000(_at_)ayalon(_dot_)eng(_dot_)tau(_dot_)ac(_dot_)il>, 
> proto=ESMTP, daemon=MTA, relay=r220-1.rz.RWTH-Aachen.DE [134.130.3.31]
> May  5 12:13:32 trei sendmail[2539]: g45JDOS02538: 
> to=<PSE-L(_at_)mail(_dot_)professional(_dot_)org>, delay=00:00:06, xdelay=00:00:05, 
> mailer=local, pri=138835, dsn=2.0.0, stat=Sent
Note in the first entry, nrcpts=1, meaning there was only one recipient, 
which is the ONLY time the "for useraddress" bit should appear in the 
Received header, or the X-Envelope-To: header will appear.  You MAY find 
that a majority of your messages contain the extra info in the header, but 
this would be because a majority of the messages had only one recipient.
A single message may be addressed to multiple people, but if the ALIASES 
resolve down to the same account, nrcpts will be one.  I sent a message 
from another host to two addresses at my host (just to eliminate 
local/v/remote handling and make it a remote message):
> May  5 13:24:29 trei sendmail[3978]: g45KOQS03978: 
> from=<user(_at_)someotherhost(_dot_)tld>, size=38943, class=0, nrcpts=1, 
> msgid=<200205052024(_dot_)g45KONg10087(_at_)someotherhost(_dot_)tld>, proto=ESMTP, 
> daemon=MTA, relay=someotherhost.tld [xxx.69.225.xx]
> May  5 13:24:30 trei sendmail[3979]: g45KOQS03978: 
> to=<somelocaluser(_at_)mail(_dot_)professional(_dot_)org>, delay=00:00:04, xdelay=00:00:01, 
> mailer=local, pri=68524, dsn=2.0.0, stat=Sent
> Received: from someotherhost.tld (someotherhost.tld [xxx.69.225.xx])
>         by trei.professional.org (8.10.2/8.11.1) with ESMTP id g45KOQS03978;
>         Sun, 5 May 2002 13:24:26 -0700
Both of the recipients resolved to the same mailbox, so nrcpts=1 even 
though the remote server addressed it to two _addresses_.  Had they been 
different users, nrcpts would have reflected the actual number of local 
PHYSICAL recipients, but the Received: line would still omit the "for ... " 
bit.  Different MTAs (or versions of same) may handle this differently - 
you should become acquainted with how your MTA operates.
If you take the messageid and look for all the lines with to=<someaddr>, 
you should have the _addresses_ to which the message was individually 
delivered to on the system, reduced to just ONE of them (IME, the FIRST 
specifed recipient) for any given aliased user:
        joe, joe.smith, bob

may resolve to just joe and bob if joe.smith is an alias for joe.

> :0 H
> * ()\<for[      <]+\/[^@        ><]+
> /var/tmp/spool/$MATCH
DANGER WILL ROBINSON!

1. You're missing the LOCKFILE for the delivery - using this when another 
message arrives (or from a 'formail -s ' operation, which can spawn 
concurrent procmail processes...) can seriously hose the mailbox.  That's 
probably even worse than having the email messages deferred.
2. At a bare minimum, you should anchor this to a Received: 
header.  However, as I point out above, it is going to be seriously 
hit-and-miss.
3. If you're running this as root (and you must be in order to deliver to 
other people's mailboxes, unless security is FUBAR on your system), this 
will create arbitrary mailboxes, Say, when TWO or more users on your system 
are subscribed to a mailing list, and the headers from that list include 
things like "Received: ... for <listname(_at_)otherplace>.  It will deliver to 
neither of the individual users, but instead create a mailbox for an 
account that doesn't even exist here.
4. It'll also be creating mailboxes which include the host/domain portion 
in the filename.
5. Delivering directly to the users' mailboxes will result in the 
.procmailrc of those users NOT being invoked (which is nearly as rude as 
having misfiled their email in the first place).  Also, the /etc/procmailrc 
won't be invoked.
6. In the case of locally hosted mail lists which alias to a program, or 
alias lists (aliases which deliver to a group), extracting the supposed 
recipient in this fashion will not re-deliver to that list (not that you'd 
want it to - and prog-delivered aliases don't invoke the LDA anyway, so the 
copy you'd have received was originally intended for an unnamed user, not 
the list).
7. I'd consider adding an 'X-Delayed: date' header to these redelivered 
messages, so that users can identify them as mangled messages (you could 
mention this when you announce that due to an administrative error, the 
mail was misfiled, and thus the redelivered mail contains this header so 
that users wishing to identify what mail was affected can do so).
8. H isn't necessary, as it is a default.  It has been mentioned several 
times before.
9. Repeat after me:

        Before filtering other people's email with a global filter, I will
        thoroughly test the filter, first in a sandbox, then as a recipe that
        logs it's actions against a COPY of the message, so that oversights
        are less likely to hose users, and thus me.

No doubt there are several other issues which other members can point out 
to you -- as a list of issues approaches double digits, it should be clear 
that the approach is seriously flawed.
>         But, in case of mail-lists I should send the message again to the
>         list server and I do not like it!
Yes, that would be bad.  How about you filter the messages to an 
intermediate set of files rather than directly to the mailspools, review 
the deliveries (AND the mailbox names), THEN process those mailboxes into 
the respective users' spools.  Do this even if you decide to implement a 
perl script like I recommend at the top - this allows you to realize you've 
screwed up before you mail loop on someone.
Needless to say, hopefully you've addressed the bad recipe, and if you 
split a mailbox file to run a script, that mailbox file should NOT be an 
active mailbox being delivered to by live recipes - it should be a moved or 
copied file.
---
 Sean B. Straw / Professional Software Engineering

 Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
 Please DO NOT carbon me on list replies.  I'll get my copy from the list.

_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail