|
Re: can procmail help me?
2004-02-04 16:18:04
At 13:24 2004-02-04 -0600, Richard WItt wrote:
I will try to explain our setup to see if procmail can help or not.
The sendmail server is just a gateway email server. It runs the
commercial version of sendmail and trend micro's interscan email
viruswall.
I am unfamiliar with how interscan is invoked, but if you can put a simple
procmail filter BEFORE THAT, you could sharply reduce processing time by
avoiding scanning the message for thousands of viral forms when a handful
of checks can spot the majority of the CURRENT problem messages (one might
hope that an intelligent virus scanner would be checking viruses in order
of their current infection rates, which would reduce the scan time when a
message is infected).
Adopting a policy of accepting absolutely no executable filetypes (EXE,
BAT, PIF, etc) can sharply reduce your processing time. Some users might
complain, but generally, the fact that they wouldn't get so many viruses in
their inboxes (or wait 14+ hours for their mail to arrive!) would likely
more than compensate. Sure, there's the ZIPfile based viruses (which is
really a social virus - it relies on someone being moronic enough to open
an attachment in a message which in all probability has come from someone
they don't really know - a single errant click won't launch them), but if
you preemptively ditch all executable types beforehand, the virus scanner
will have a lot less to do. The current crop of viruses is certainly easy
enough to identify in procmail.
You should review how interscan is being invoked and see whether procmail
might be able to be invoked in its place, say with procmail invoking the
scanner only if the message wasn't already discarded by a simple check in
procmail, or isn't larger than some size (plaintext and < 5KB messages
probably aren't infection sources, but that's a judgement call you'll have
to make). It's easy enough to change the procmail logic later if a
situation crops up that calls for scanning _everything_.
It is also running a spam proxy daemon that passes emails
from sendmail to spamassassin then back to sendmail.
Yea, this sounds _really_ efficient....
The reason i need help with procmail is because there is no local delivery
at all.
Typically a stumbling point, because procmail is usually used for LDA
only. However, you can invoke it if you're able to invoke something like a
virus scanner _program_ (i.e. if the virus scanner is run for each received
message, not already running as a daemon).
After the scanning for viruses and spam happens all email is then forwarded
to our internal exchange servers.
Heh. You should see how Exchange would respond to the load if it were
front and centre... <g>
being sent back to spoofed addresses at our domain.
Well, invalid addresses should bounce pretty promptly (and quite a few of
the virus bounces do bounce back to bogus usernames at the domain because
they were forged that way). Some A/V programs use a specific username
portion in the envelope sender address, which you could set up in sendmail
to be rejected, seeing as the vast majority of these systems respond in
this manner to viruses which are known to use forgery. By bailing at the
SMTP envelope exchange, you avoid receiving the message body and having to
subsequently invoke any of your scanning processes, which means you shed
load quickly. Same concept as running a DNSBL of infected hosts (which
which a bit of code could be automatically populated from the output of the
virus scanner).
Right now our emails are being delayed by 14 hours.
Sucks to be you.
Sounds like a great job opportunity though, if companies weren't generally
too paranoid about using hired guns to solve their emergencies.
I am not the email admin but work very closely with him, and he tells me
thats because our primary email server is busy so emails coming in
fail over to the secondary email box.. which if its busy.. the
incoming request just waits to keep trying to send to the secondary
and doesnt try to resend to the primary first.
For the time being, you may be MUCH better off removing the secondary from
your configuration. For one, I doubt the micro SMTP server setup in the
viruses requeues for delivery, so if you manage to avoid the message
because your primary server is too loaded, you won't end up having it
handed off to the backup server which will continue to harrass the primary
(assuming the virus follow the MX chain). Also, it's unlikely that the
backup MX does any filtering, so it's gleefully accepting ALL messages,
which only compounds problems for your primary MX.
Not that I'd normally recommend anyone run without a backup MX, but any
legitimate sending host should handle requeueing of the message to attempt
sending again later if they cannot get through (though there are some
wonderfully lousy configurations out there that try one send and then
bounce if it didn't deliver). Certainly, you should be no worse off than
you are now, since your backup MX is effectively condtributing to a DoS
against your primary MX.
If the backup MX is under your control, the thing to do would be to
TEMPORARILY firewall SMTP connections from any host other than the backup
from your primary MX, and similarly firewall the secondary host (presuming
it's a backup MX decated to your domain(s), but even so, a temporary
firewall isn't going to toss any mail except for questionable stuff). Your
primary should then be able to complete processing of the mail in its local
queue unhindered by outside influences, and during this time, the secondary
should attempt queuing of the messages it has (or, when the process load /
connection count on the primary has stabilised, you can issue an ETRN to
the secondary, which will force a queue flush). Once the messages from the
backup MX have been transferred, update your DNS to remove the backup MX
from the mix (again, this is a temporary thing, at least until you have
your filtering sorted). You should flush the named on the backup MX (if
there is one), or in the meantime, configure it to reject messages being
delivered to your domain (with a tempfail, so the sender will attempt a
requeue later). Then, remove the SMTP firewalling from the backup and
primary SMTP hosts. External connections to the backup MX should be
rejected (since that host should now recognize it isn't handling mail for
your domain), and the primary MX should be accepting connections up to it's
configured limit, but generally NOT requeued viral messages (this doesn't
mean you won't still be receiving viruses, just that infected hosts
shouldn't manage to fill the backup MX with spew as well).
If I were in your situation, I'd isolate the more consistent senders of
viruses (as demonstrated on this very list a few days ago, one host can
spew quite a bit), and add their IPs to a locally-administered DNSBL. If
you're not operating your own DNS, you can still do a basic form of this
using the access db feature of Sendmail:
CONNECT:w.x.y.z REJECT
This alone can spare you from a lot of the cruft, which may allow your
regular system to keep pace, even though some number of viruses will
continue to spool in. Note that this WILL NOT protect you against viruses
coming in via your backup MX (unless it's configured to use the same list
of infected hosts), since it'll otherwise accept messages, and then when it
relays them to you, the connection address will be your backup MX, not the
infected host.
As for whether procmail can help you - as has already been pointed out, if
you're already getting hammered by the _connections_, and you're running
other processes which load the system, procmail isn't going to stop any of
that because it can't process the message until AFTER the SMTP
session. OTOH, if the system setup permits invoking procmail as a front
end to your virus scanner, you can probably save yourself a lot of CPU
time, which results in ditching the problem messages sooner, which will
result in a lower CPU (and memory) load.
If your mail admin decides a hired gun can help, contact me offlist. The
preceeding however is the extent of my free advice.
---
Sean B. Straw / Professional Software Engineering
Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
Please DO NOT carbon me on list replies. I'll get my copy from the list.
_______________________________________________
procmail mailing list
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)DE
http://MailMan.RWTH-Aachen.DE/mailman/listinfo/procmail
|
|