spf-discuss
[Top] [All Lists]

Re: Possible SPF machine-domain loophole???

2004-02-25 03:09:30

----- Original Message ----- 
From: "Greg Connor" <gconnor(_at_)nekodojo(_dot_)org>
To: <spf-discuss(_at_)v2(_dot_)listbox(_dot_)com>
Sent: Wednesday, February 25, 2004 1:29 AM
Subject: Re: [spf-discuss] Possible SPF machine-domain loophole???


OK, here is some quick feedback... rather than replying to each message in
turn I will combine my thoughts so far on this thread.


Everyone:  Dang, you wrote a lot of mail in one day!

Hector:  You have NOT shown that this is a loophole.  Please don't
describe
it as a "loophole" or "flaw" -- there is a good reason to limit these
checks that has to do with limiting any potential losses of real, desired
mail (such as the cooking.com example Meng mentioned).  If you believe I
am
fundamentally missing something, it's OK to write me privately.  I will
accept remedial lessons and if I change my mind I will admit it to
everyone.  :)

I'm sorry, I prefer not go to off-list and repeat everything.  I believe
very confidently I did show it with an actual example. It is really quite
trivial:

example #1 - no SPF record with SPF-none result

        client IP: 1.2.3.4
        HELO  winserver.com
        mail from: user(_at_)example(_dot_)com

example #2 - SPF record with SPF-pass result

        client IP: 207.8.214.5
        HELO  winserver.com
        mail from: user(_at_)aol(_dot_)com

example #2 - SPF record with SPF- softfail or neutral result

        client IP: 202.178.165.242
        HELO  winserver.com
        mail from: user(_at_)aol(_dot_)com

In each case, SPF would by pass the spoofing of the winserver.com domain.
That is not a badly configured server, but one that is maliciously spoofing
the helo domain by using our local domain, winserver.com

The DMP logic checks for this.  SPF logic does not.  Thats a SPF loophole
and it is not consistent with the LMAP guidelines and it is also
inconsistent with Meng's description of SPF has being the merger of DMP and
RMX.  It can't be a merger of DMP if it breaks this fundamental requirement.

Thats a loophole and if SPF does not address, it forces the MTA software to
perform this check which defeats the purpose of SPF.

Ernesto:  You have a good point, and so does Hector, that being that we
*could* validate the HELO name at other times than when the spec says.
But, it's important to listen to what Meng points out - many hosts will
have HELO configured wrong.

I don't agree with this.   There is traditonally 3, actually 4 reasons why
the SMTP specifications has relaxed the HELO specification.

1) MUA software uses the NETBIOS domain name, not a FDQN.

2) Large Outgoing farms or send only machines may be configured to use
single setup with one main domain name, not one that resolves back to the
sendering machine.

3) Many high load systems turn off rDNS for performance reasons.

4) To provide a minimum transaction for system notification to report bad
configurations or setups.

#1 is still a problem today and the recommendation for future systems is use
a FQDN.

#3 is still common

#4 is the original reason for the relaxation, doesn't apply today.  There
are other ways to do contact.

#2 is an old problem,  today with modern SMTP servers, they are doing it
correctly, EXCEPT for spoofers!

If you are checking envelope sender all the time, and only checking HELO
when the envelope sender is blank, then the hosts with mis-configured HELO
will still be able to send you legitimate, non-forged mail.  If you limit
the HELO check to MAIL FROM: <> then you are limiting the scope of the
false-positive problem to 1. bounces from badly-configured servers, but
still allowing legitimate mail from those servers, and 2. suspicious
messages not following SMTP guidelines.  I think it is a mistake to tell
people to apply HELO checks to ALL mail -- yet.  (BUT, read on for what I
am actually suggesting).

I have said repeatedly, that SPF should only check for HELO/EHLO local
domain spoofing.  Checking for external domain spoofing should be a system
implementation option.

1. Allow receivers to check the HELO name.  They MAY check the HELO.
(After everyone likes it and agrees it is a good idea, we can change the
spec to SHOULD check, but we probably should try not to piss off anyone
who
has already implemented it.)

I disagee.  Lets not start a NEW spec that hasn't been yet endorsed as
standard to begin with kludges.  It should be a MUST for Local Domain Spoof
checks, a "MAY or SHOULD" for remote domain spoof checks.    If everyone did
a local spoof check only, then as a total network, the end result would be
optimal with the same results.

Keep in mind this:

         MUA --->  MTA1 ---> MTA2 --> MTA3 ---> Final Destination.

When everything is said and done in a widely deployed system, the trust
throught the route will be established.

Transaction MUA-MTA2  must be authenticated for the route otherwise MTA1 is
an open relay.

Transaction MTA1-MTA2  must be authenticated for the route otherwise MTA2 is
an open relay.

Transaction MTA2-MTA3  is allowed as anonymous access since it a final
destination.

In other words, for transactions MUA-MTA1 and MTA1-MTA2, there is no need
for "validation" because the session is already trusted/authenticated.

SPF is breaks down with the MTA2-MTA3 transaction unless MTA2 is allowed to
send mail on behalf the MTA1 domain network. SRS attempts to address this.
Reading the Received: lines can also be used.

But regardless, if you want to use SPF at MTA3 and attempt to create an
"enforcement policy" for MTA1, then you must also require MTA2 to use a
valid FQDN HELO domain.

In other words, you can't have it both ways.

        - MTA1 must comply with the MAIL FROM:
        - However, MTA2 doesn't have to comply?

Something wrong with that picture.   If MTA2 is a spoofer, then the odds are
really high that MTA1 is a spoofer too.

SPF only works if the "links" in the chain are consistent.

With that said,  of course, a system administrator should be able to have
the option to perform whatever checks he wants.   But if the default is for
SPF not to perform at a minimum a local domain spoof check, then you have
defeated the purpose of what we are all trying to address.

By far, most people will be more concern "first" with protecting their
system and the malicious and criminal fraudulent usage of your domains and
address.   So local domain spoof protection is the #1 benefit all the LMAP
proposals offer and this will work 100% all the time.

Also, even if thre is a False Negative (rejection), if the system is serious
in trying to send mail, then they will try again or contact you one way or
another.   But by far, it is emperical fact that over 80% of the spammers
are spoofers and they will not try again to send the same message
transaction.   The little possible false negatives is absolutely worth the
added enforcement and accountability that LMAP brings to you.

This is all very easy.  Check for Local Domain Spoofing.  That is the
immediate benefit that LMAP proposals offers and it works 100% of the time
except for SPF with this non-null MAIL FROM loophole.  DMP addresses it.
SPF should too.

I have not stated whether an SPF record has to be used for helo domains.
What I am saying is  the "functional specification" needs to have it written
in stone that local domain spoof checking is a requirement otherwise the SPF
benefits are lost..  SPF should offer the logic, but it should also be an
administration option if the MTA already has this logic built-in.

If SPF logic is to be used for HELO domains, then it should be for the base
domain.

Otherwise SPF algorythm, APIs, liberies, etc, should simple A record look
ups.

Examples (actual, real transactions logged):

Example #1

        client ip: 24.151.134.154
        HELO cesmec.cl
        MAIL FROM: <lestercardenasqt(_at_)aol(_dot_)com>

SPF will return a SPF-neutral for the mail from: aol.com domain.

An A record lookup for "cesmec.cl" returns 200.68.8.51.   A remote domain
spoof or bad configured system?.
Can a system couple the neutral result with the helo spoof and change the
overall result to softfail or fail?

Regardless, SPF has no provision to check for the HELO since we have a
non-null return path.

Example #2

        client ip: 24.151.134.154
        HELO cesmec.cl
        MAIL FROM: <>

SPF will return a SPF-none for the domain cesmec.cl

Again, with no logic to check for the HELO domain spoofing, in both
examples,  you have a successful SPF operation as far as continueing to
allow the transaction to move on.  In one case, you have a neutral and the
other a none, when in reality,  the odds are very very high the IP/domain
mismatch suggests that it is a truly a spoof with a confidence level that is
<= 100%.

Now, change the HELO domain to a local domain that you own, like my
mail.winserver.com

Example #1

        client ip: 24.151.134.154
        HELO mail.winserver.com
        MAIL FROM: <lestercardenasqt(_at_)aol(_dot_)com>

Example #2

        client ip: 24.151.134.154
        HELO mail.winserver.com
        MAIL FROM: <>

In both cases, these are 100% spoofs with 100% confidence.

SPF will fail in both cases with a SPF lookup on mail.winserver.com.   For
systems with 100 domains, you don't want to revert to having 100 SPF records
like DMP required.  This is one of the reasons why Meng wrote SPF in the
first place, to avoid all the records as required by DMP to cover all bases.

So SPF will only trap #2 if a SPF lookup on winserver.com, not
mail.winserver.com.

In both cases,  SPF will *not* trap #1 - hence your loophole in both cases.

So as I suggested to Meng, the SPF lookup logic should place a higher
emphasis for local domain spoofing since the result has a 100% trust value.
However, you have <= 100% when checking for remote domains.  So remote helo
domain checking should be an option.  Not a requirement by default.   Only
local domain spoof checking should be a requirement by default.

Would this suggestion satisfy people?

You don't have to satisfy me.  I'm already convince what is the flaw and
what needs to be done - not for me, but as a product for over 100,000
customers who will use our system.    They will not have the problem.

What concerns me is that I don't want to see what happen to SMTP with
"loopholes" and "relaxed" logics that causes the problem in the first place.
Now is not the time to have "kludges" and "loopholes" in a specification
that has barely made it out the door.

And I 100% understand the position Meng is in.  If the SPF specifications
requires the SMTP server to perform the HELO checking, this is the kind of
thing that might hurt its endorsement.   But maybe as long as it written in
stone as a MTA requirement for SPF operations, it might address the concerns
like many here seem to feel.  That's fine by me.

But I am not sure with the IETF people will feel about having the MTA
required to do HELO spoof checking in order for a SPF "RFC" to be workable.
Sounds kind of kludgy to me and its not even a standard yet!

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com