From: wayne
Sent: Sunday, April 18, 2004 1:35 PM
In <MHEGIFHMACFNNIMMBACAEEDAHMAA(_dot_)sethg(_at_)GoodmanAssociates(_dot_)com>
"Seth Goodman" <sethg(_at_)GoodmanAssociates(_dot_)com> writes:
With regard to HTTP, I'd be surprised if the cost of a CBV to
the message
recipient MTA is nearly as high as anything based on HTTP. While it is
certainly more expensive than a UDP DNS query, it is about as
short an SMTP
session as is possible.
Erh?
Both HTTP and SMTP use TCP/IP, so they both require a three-way
handshake on connection create and a four handshake for closing the
connection. The HTTP requires one command to be sent (one packet) and
one response packet to be recieved (on packet). A CBV via SMTP is
going to require exchanges for the HELO, MAIL FROM: and RCPT TO
commands. The receiving SMTP server will also likely do a bunch of
other DNS checks to filter out the spam sources.
My guess for SPF using UDP-DNS vs TCP-HTTP was a factor of 10. Others
said it is likely higher, James measured it at over a factor of 9.
My guess is that CBV is at least a factor of 2 or maybe even 3 more
than HTTP.
I'm afraid you are leaving out the possibility that multiple requests may
well be needed in an HTTP environment. DNS is built for recursion, while I
don't believe HTTP can do this easily.
SES returns a binary result, pass or fail, from the
domain MX, which
the domain owner generally has more control over than SPF
implementations at
other sites.
Again, no SES+CBV does *not* return a binary pass-fail result. It
returns an indeterminate-fail result.
Perhaps you should review this, but SES+CBV returns pass/fail. I really
have no idea where the indeterminate result that you mention comes from.
Certainly not from anything I proposed. If the crypto cookie does not
validate, the result is fail, not indeterminate. It says authoritatively
that the message did not originate from that MTA farm.
This is part of what I meant by "you get what you pay for". A
CBV is more
expensive than a DNS lookup, though as James Couzens described in his
outline of SPF processing, there is quite a bit more logic (CPU
cycles) and
often additional DNS lookups, so it is not simply issuing a
single DNS query
and getting a yes/no answer.
Several points:
First, the number of SPF DNS lookups is not the same as the number of
DNS packets on the net. Many parts of the SPF checking will have DNS
cache hits because information about the domain's A and MX records
will likely be checked by the MTA anyway. Specialized DNS caches that
understand the DNS usage of SPF can reduce this further.
You are assuming changes to the existing DNS system or special local DNS
implementations. AFAIK, querying for A and MX records will not cause TXT
records to appear in the cache. Even if they did, the queries would still
have to occur. Queries for unique, non-existent domains seen in spam will
certainly not be cached.
As far as CPU costs go, the CBV is going to be more expensive than
SPF. I did some quick profiling on libspf-alt a while back, and the
largest use of CPU was the "sanitize" function. It is the function
that makes one pass over all of the output to make sure there aren't
any invalid characters.
This, at first, surprised me, but after thinking about it, I realized,
for the most part, everything in libspf-alt is going to be
approximately a linear function of the bytes exchanged across the net
(e.g. SPF TXT record, DNS lookups, etc.). Nothing that libspf-alt is
a huge amount more expensive than calling the very fast isprint() C
function, hence the sanitize() function, which makes effectively
several additional passes over the data, will rank very high. (It has
to do one pass for the SMTP comment, one for the Received-SPF: header
comment and one for the Received-SPF: header itself.)
CBV is going to have very similar CPU usage since it also has to
scan/checksum/parse all the data sent across the net.
Therefore, since CBV uses more network resources, it will almost
certainly use more CPU cycles.
As I stated in the post you are responding to, the incremental _total_ load
for both network bandwidth and CPU cycles in a real mailer will likely be
negligible for either SPF or SES. Since you don't mind splitting hairs, SPF
requires SRS, which requires a hash calculation with every forward. That is
certainly an extra CPU load. Even with that, I would still guess that the
overhead for any of these schemes will be dwarfed by the network bandwidth
and CPU cycles used in receiving the DATA part of valid messages.
6) SES does not require changes to RFC2821. SRS violates RFC2821 by
changing the return-path in transit and requires changes to
RFC2821 to be compliant.
I think you're probably right on this one, too.
I think the difficulty of this one has been underestimated or largely
ignored.
Please explain why you think that SRS violates RFC2821. As I said, in
the IETF WG studying this stuff, there doesn't seem to be any such
consensus. This is an issue that I consider very important. It has
certainly not been ignored by me, or by quite a few others.
We all should consider this very important. I would like to refer this
question to David Woodhouse. While I have an opinion based on my reading of
the standard, I am not as expert as either you or David on this, so I invite
him to express his opinion.
Using SES (that is, applying SRS to all outgoing email) lets you
immediately start rejecting bogus bounces that are being sent back to
you, which is great. I like that. It doesn't allow the receiving MTA
to check the envelope-from without an expensive CBV.
If the CBV's don't change the _total_ load on a real mailer (network
bandwidth and CPU cycles) by a significant amount, then the fact that a CBV
is expensive when viewed in isolation is not relevant.
I see no reason not to use both SES and SPF, along with something like
MS's Caller-ID, or Yahoo's DomainKeys, or S/MINE or something to
verify the mail headers. These tools all allow you to solve different
problems, but they don't do a good job of solving every problem.
I agree that all these systems could be used together and are interoperable.
However, SES does everything SPF does without incurring the breakage of SRS.
That alone is a huge advantage.
--
Seth Goodman