ietf-mxcomp
[Top] [All Lists]

Re: Unified SPF: block versus factored records for HELO and MTAMA ark scopes

2004-06-24 22:06:19


On Jun 25, 2004, at 12:30 AM, Hallam-Baker, Phillip wrote:
Most Unix MTAs including sendmail, exim, qmail and (I believe) postfix
fork off a process per SMTP session which exits at the end of the
session.  In this case, any effort beyond that to validate the single
IP used in the session is wasted, and the local DNS cache is where the
info will be remembered.  My impression is that there are a lot of
large sites whose MTAs work this way.

I do not know what the situation is where dedicated edge servers
such as spam filtering systems are concerned. I strongly suspect
that these are based on very different architectures. Perhaps someone
could go and look at what some of these systems do.

Our Edge appliance (and Ecelerity MTA software) does not use a per process model. It actually doesn't use a per thread model either. It is a hybrid model where non-blocking ops are put into a small group of event engine threads and blocking ops are put into a larger group of worker threads.

As DNS queries have no good reason to block (there are a handlful of free nonblocking DNS resolving libraries out there) we can realize high concurrency and throughput with DNSBL, SPF, DomainKeys, etc. all performed inline. 100,000 concurrent SMTP sessions all performing DNS based lookups, MySQL lookups, ldap lookups, virus-checks and a slew of other things in real-time (during the SMTP session).

Even on a UNIX box there are good reasons to prefer a per thread model
to a per process model, process creation and teardown in UNIX is only
lightweight compared to other O/S.

Threads have costs too. Smaller than processes in most cases and almost negligible if they are user-space threads. However, each has its pros and cons and you have to carry around that thread-stack. So one thread-per connection can be a big big waste.

There are also other ways that a UNIX system may be organized. For
example if I was operating a large cluster of mail filters I would
hive off the task of fetching MARID data and resolving all forms
of reputation and accreditation data in a separate system that
performed caching.

Yes... perhaps not "separate", but certainly cached.

I would not want to store state in the mail server itself for the
reason you mention and also because that would load up my external
connections (which I pay for) rather than my internal 1Gb ethernet
connections which are essentially free.

You can used a replicated data store. Shared consistent replicated cache across the mail servers. Much better and more reliable than a "backend store" as they usually have to be over-engineered to avoid a single point of failure.

// Theo Schlossnagle
// Principal Engineer -- http://www.omniti.com/~jesus/
// OmniTI Computer Consulting, Inc. -- http://www.omniti.com/
// Ecelerity: fastest MTA on Earth


<Prev in Thread] Current Thread [Next in Thread>