RE: a grand unified theory of MARID (blame me!)

Hello from a regular MARID poster!

Meng's latest SPF plan (in which he has borrowed my idea of CSV 
semantics using SPF records) was influenced by, among other things, a 
group of us pushing strongly for HELO to always be checked.  SPF already 
required that the HELO be valid, so the change from saying that it MAY 
be checked to it MUST be checked is not an issue, with respect to 
senders having to comply with a new requirement.  Meng's latest SPF 
plan, does not require senders to comply with a new requirement at all 
(vs. the old SPF).  In fact, it's now EASIER for senders to comply.  
MUCH easier.  This is why I, at least, pushed hard for the CSV solution 
(which I'm no longer going to push, because Meng's latest SPF plan does 
what I've had a burning desire to see MARID do.  Let me explain.
I'm posting to explain WHY this is a good thing and HOW it will make 
things EASIER.
I think a HELO check accomplishes a lot more than is immediately apparent.
I think a HELO check can protect 2822.From:, and with a major tweak, 
prevent Joe-job damage, while being much easier to implement broadly 
than other identity checks, and accomplish the goal of tying email 
better to a responsible party, bringing us much closer to a realistic 
near-FUSSP.  Detailed arguments below.
First, a quote from Dave Crocker (2822 author):

   Breaking legitimate functionality of a system that has been in use for
   30 years and is currently relied on by 1 billion people obligates
   those doing the breaking to be very clear and careful about defining
   and defending the breakage. That is not happening about this topic.



Remember, whatever the source for the domain that MARID validates, 
blacklists and reputation services will be an integral part of the system.
A domain with a MARID record used in email with malicious forged From: 
is gonna get in an RHSBL lickety-split:
If you get word of From: forgery, you're gonna be motivated to do a 
little work to get the spammer's domain blacklisted, for example by 
putting him in your RHSDRBL (Right Hand Side Distributed RBL), which 
will stop the forgery and phishing.

If we ensure that HELO passes:
Can a spammer set up a domain and rDNS with records under the spec and 
spoof 2822.From: yes, for all the extant I-Ds, including SPF, and C-ID, 
(remember, Resent-From, etc. can be abused) BUT not for long - the 
authorizing domain will get blacklisted PDQ.
Is a spammer forced to use a domain set up with records that specify its 
authorized MTAs: yeah.
Are forwarders or remailers' systems broken, requiring new software: no 
for unified SPF (i.e. with mandatory HELO checks), yes for SPF as it was 
yes for C-ID (PRA/RFrom). 
Are users forced to send through MTAs authorized for the domain they use 
in outgoing mail: no for unified SPF, yes for SPF, C-ID, Y!DK (even 
stricter!).
MTA admins just need to ensure that they are sending appropriate HELO 
strings, which nearly all already are, and create DNS records. Wow, 
that's not a lot of systems that need touching, relatively speaking! 
(Around 2 orders of magnitude fewer than SPF+SRS).
ALL the other proposals so far don't accomplish much more than just the 
HELO check, and yet do impose much larger costs.
It ties to domain owner info, instead of the less accurate IP space 
owner info. Nice, but both are often false.
Now that it uses SPF records, it has nifty flexibility and extensions in 
specifying authorized IPs.
Y!DK is even more restrictive (and disruptive), as the keys can only be 
placed on a much smaller subset of machines than SPF could authorize, 
the only benefit being it defends against use of bogon or hijacked IP 
space better.
A mailing list post with a forged From proved a point rather well: No 
proposal with an I-D listed in the MARID charter would have completely 
prevented the forgery from making it to the list. So if they can't give 
the list-management software the tools it needs to prevent the forgery, 
what does all the additional adoption headache they cause buy us? In 
fact, there's an argument to be made that with HELO, SPF will do a 
better job preventing the forgery; it goes like this:
Because it will have a much lower implementation cost, it will get 
implemented sooner. After broad adoption, this will be tough: the forger 
will have just two major options: use an un-blacklisted domain he runs 
on an MTA he runs (losing anonymity), or free rein over these things run 
by someone who controls them well enough to keep the domain off BLs (an 
increasingly hard thing to find).
PS Yes, my views on the criticality of MARID directly and fully 
protecting 2822.From have changed - I didn't see that there was an 
alternative way to protect it.
If you say HELO checks fail to protect 2822.From:, I have two 
counter-arguments:
1) But SPF fails to protect the domain in From: too.  And if we say 
that's OK, it protects From/Sender/Resent*, then we're saying SPF 
doesn't protect From 'till everyone upgrades to mail clients that 
support this.  Now we're talking what?  Easily 2 orders of magnitude 
more work than EHLO checks. 
2) Not to mention that I explained above how CSV does provide a way to 
protect From: (through incenting blacklist submission).

Initially, I was pushing for CSV mainly because it checked HELO, and I 
had strongly felt ideas about why a HELO check was the best thing to check.
The above is a rephrasing of this post:
http://www.imc.org/ietf-mxcomp/mail-archive/msg01175.html and this post:
http://www.imc.org/ietf-mxcomp/mail-archive/msg01226.html

Unified SPF makes sense because it economizes on human time.  The 
algorithm is a bit more complex, but only needs to be written once (or a 
few times) and breaks much less legitimate functionality without leaving 
more illegitimate functionality working than SPF's previous version. In 
return, it's MUCH easier to deploy, because it requires NO end user 
action, and eliminates the need for SRS!  That's an amazing trade! 

I welcome your questions and comments.

=-=-=-=-=-=-=

Greg Conner said:

> Layer 0: HELO and PTR
> These should bind strongly to IP. Unfortunately there are cases where the
> HELO is a fake name, or the PTR is missing, or both. At this stage we are
> looking for a FAIL, in order to spot obvious forgeries like "HELO
> microsoft.com" or HELO with the receiver's domain as many viruses do. 
> This
> catches some of the "low-hanging fruits" of the forgery world. By 
> itself a
> PASS result here doesn't validate the whole email, it's just the bare 
> minimum
> check for an MTA. 
I don't think the implication here is valid.  By themselves, Layer 1 or 
Layer 2 don't validate the email EITHER.
They must be used in conjunction with a reputation service - remember, 
spammers are already sending spam that  gets an SPF "pass" w/o a 
reputation check.  When HELO is used in conjunction with an RHSBL and 
RHSWL or a reputation service, the "pass" DOES validate the whole email.
What I like about this new thrust is that the reputation service 
application to each identity is explicit.