procmail
[Top] [All Lists]

Not scaling well

2003-09-15 09:33:25
Wondering if anyone sees any inefficiencies in this beast. I've trolled the FAQ's and message boards, but just seem to break it whenever I try to improve it. It worked fine for 250 users for a number of years, but now we're pushing a 1000 and, as we all know, the level of spam has skyrocketed. It's running globally out of the main .procmailrc file on a QMail-LDAP server on FreeBSD 4.6. Watching top, it's just filled with 'procmail' and 'egrep' processes chewing up all available CPU. The user defined white and blacklists contain various possible entries including 'spammer(_at_)test(_dot_)com', 'test.com', etc,etc all the way down to single word matching like 'spammer' which would catch it anywhere in the address. A few users have lists with several hundred entries.

We originally found this script on a procmail FAQ/examples page somewhere, and it's worked great, but now we're hoping we can avoid throwing faster hardware at the problem. Searching around I know there has to be a better way, but can't seem to come up with the magic incantation. Hoping someone out there might see something in there and be able to lend a hand.

Thanks in advance!


# Test if the email's sender is in user definded whitelist, if so deliver it.
:0
* ? formail -x"From" -x"From:" -x"Sender:" -x"Reply-To:" -x"Return-Path:" -x"To:" | egrep -is -f /usr/local/apache/htdocs/secure/usermaint/nobounce/${USER}
${MY_MAILDIR}

# Test if the email's sender is in user definded blacklisted
# if so, send it to back to sender w/ bogus user unknown
# mark with "Recipient Refusal" so it can be traced back

#Define getting the sender's address, Discard any leading and trailing whitespaces
FROM_=`formail -rt -xTo: \
  | expand | sed -e 's/^[ ]*//g' -e 's/[ ]*$//g'`
#Return certain blacklisted email
:0
* ? formail -x"From" -x"From:" -x"Sender:" -x"Reply-To:" -x"Return-Path:" -x"To:" | egrep -is -f /usr/local/apache/htdocs/secure/usermaint/blacklist/${USER}
# Avoid forgeries that pretend to be from my own site
* ! $ ? echo ${FROM_} | fgrep -is 'boothcreek.com'
* $ ? echo ${FROM_} | fgrep -is '.'
* $ ? echo ${FROM_} | fgrep -is '@'
# Avoid email loops
* ! ^X-Loop: postmaster(_at_)mydomain\(_dot_)com
{
  # Make a temporary file of the message to be returned
  :0c:formail.lock
  # Discard whitespaces, insert a leading blank
  | expand | sed -e 's/[ ]*$//g' | sed -e 's/^/ /' > return.tmp
  # Prepare and send the rejection
  :0:formail.lock
  | (formail -r -I"Subject: Rejected mail: Recipient refusal" \
    -I"From: ${ALTFROM}" \
    -I"Return-Path: noreply(_at_)mydomain(_dot_)com" \
    -A"X-Loop: postmaster(_at_)mydomain(_dot_)com" ; \
    echo "" ; \
echo " This user has choosen not to receive emails from this address." ; \
    echo "    Please contact them in a different manner (#5.1.1)" ; \
    echo "  " ; \
    echo "--- below is a copy of the rejected mail ---" ; \
    echo " " ; \
    cat return.tmp ; \
    echo "--- end rejected mail ---" ; \
    rm -f return.tmp) \
    | /usr/sbin/sendmail -t
}




_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail

<Prev in Thread] Current Thread [Next in Thread>