Re: a grand unified theory of MARID

Meng,

On 444 lines you have explained something about "fusion". You are 
talking about RMX, DMP and SPF, about CallerID, SenderID and DomainKeys. 
There are also worlds like MTAMark/SS, DRIP and CSV.
I was interessed in SPF because it is very simple. And because it was 
possible for me to implement the parts that were not implemented (I have 
written the sendmail-patch for libspf and I have corrected many bugs in 
libspf). And now I have exactly that what I want and what I need.
In your text here you have used many things that I've never seen and 
heard before (MTAMark/SS, DRIP and CSV). So I was not able to follow all 
your ideas. And I wount be able to implement all you explained here. I'm 
woundering if I'm the only one who does not understand all thes things 
exactly. I think no. So my questions to you:
What do you want?
Where do you want to go?
Who should come with you?

I'm going to use a simple thing to prevent the forgery of 
E-Mail-Addresses. I'm going to use a thing that thousends of admins 
already have deployed. You would call it "SPF Classic". Me in my 
position I do not have time, need, monay and nerves to program a more 
complex solution. Why should I? Because you would be happy if you can 
say my peoples have all implemented and it is running. No, for me this 
is no reason.
Even if you continue your way to a super complex thing that noone can 
administrate (and perhaps noone can implement correctly), I wish you all 
the best on your way, but I won't come with you. I don't need it.
Teddy


Meng Weng Wong wrote:

> I have a funny feeling that MARID will eventually engender a
> fusion of the proposals that are now on the table.  And when
> I say "fusion", I mean the same kind of "fusion" that puts
> on a single plate spicy spring rolls with creme fraiche and
> smoked salmon drizzled with a curry Bearnaise sauce.
>
> But that kind of fusion is ultimately okay.  Maybe it'll
> taste good!
>
> I expect the same to happen with the work we are doing.
>
> Let me explain.
>
>                            * * *
>
>                       FOUR IDENTITIES
>
> So far we've seen four schools of channel-based
> authentication, each focusing on a different identity:
>
>   - the IP address and its related PTR record
>   - the HELO domain
>   - the return-path
>   - the Purported Responsible Address from the headers
>
> It doesn't have to be winner-takes-all.  I think sensible
> people will find a way to get useful work out of each
> identity.
>
>                            * * *
>
>                   ACCEPTANCE AND REJECTION
>
> There are acceptance-oriented and rejection-oriented ways to
> scrutinize incoming email.
>
> An acceptance orientation lets a receiver say "oh, I
> recognize the sender; they're trusted.  I'll bypass further
> checks."
>
> A rejection orientation lets a receiver say, "hm, there's
> something fishy about this; I might reject them, or I might
> subject them to a biased scrutiny."
>
> We have seen, to date, that the designated sender components
> of SPF make it possible to perform channel-based tests on
> the three domain-related identities above.  But the overall
> framework of the SPF syntax makes it possible to do
> content-based tests as well --- you can define "pgp",
> "smime", "dk" and such mechanisms.  Just keep that in mind;
> it'll be useful later.
>
>
>                            * * *
>
>             A brief note about SPF and SenderID.
>
> I support SenderID and believe it will provide value.
>
> However, I have chosen to use the name "SPF" in this email
> in a lot of places.  This is for technical reasons.
>
> When I use the term "SPF", I am referring to the function
> that returns an authentication result based on a tuple of IP
> address and domain name.  (This is for DS.  For DK, it
> provides a link, and other code has to handle the crypto.)
>
> When this function is applied to the return-path, we have
> "SPF Classic"; when it is applied to the PRA from the
> headers, it is "SenderID".  When it is applied to other
> identities, it becomes other things.  More on that later.
>
>
>                            * * *
>
>            WHAT THE PROPOSALS WANT SENDERS TO DO
>
> Each proposal chooses an identity and asks sending sites to
> slap new DNS records on that chosen identity.
>
>   In MTAMark and SS type proposals, the records go on the IP
>   address's PTR.
>
>   In DRIP and in Dave Crocker's CSV proposal, the records go
>   onto the HELO domain name.
>
>   In SPF Classic, the records go on the return-path domain
>   name and the HELO domains too.
>
>   In CallerID and SenderID, the records go on the PRA
>   domain.
>
>   In DomainKeys, the records contain public keys, and work
>   for the "From:" identity.
>
>                            * * *
>
>                SUPPOSE WE DO ALL OF THE ABOVE
>
> Let's suppose a well-respected sender decides to play it
> safe and do all of the above.  It slaps SPF records on the
> HELO domain and the return-path domain.  It applies to
> trusted-forwarder.org to get its IP addresses listed.  It
> very carefully prepends the right header in mail so the PRA
> algorithm returns the right result.  It even signs its
> messages using DomainKeys.  Let's further suppose that
> sender has acquired accreditation credentials, and enjoys a
> good reputation in third-party reputation systems also.  So
> it's got all its bases covered.
>
> When a receiver sees a connection coming from that sender,
> it has a surfeit of identities it can use.
>
> It could look up the IP address in trusted-forwarder.org.
> If the IP address is found, the receiver could say "okay,
> it's trusted; we'll whitelist it based on IP."
>
> But if the IP wasn't found, the receiver could look up the
> HELO name that identifies the sending server.  The receiver
> performs SPF checks on the helo domain and confirms that the
> use of the name is authorized by that name.  If the name is
> thereby authenticated, and if the name is known (via some
> external lookup into an accreditation or reputation system)
> to be trusted, then the acceptance orientation kicks in, and
> the receiver says "okay, it's trusted; we'll whitelist it
> based on who the server is."  I'm thinking of taxis lined up
> at the airport, all trusting that they'll get a good fare
> simply because the customer's walking out of a certain door.
>
> If the HELO server isn't trusted, the receiver could look up
> the identity in the return-path.  Check the SPF on that, and
> again the receiver gets to play the acceptance / rejection
> game.
>
> Next the header "Sender" (and others from CallerID).
>
> Finally the "From" if the message was cryptographically
> signed.
>
> Sensible senders will want to make life easier for receivers
> by covering all their bases.  Sensible receivers will use
> all the inputs available to make the best decision.
> This is an important principle we should keep in mind.
>
>
>                            * * *
>
>        TWO LAYERS OF TESTS: AUTHENTICATION AND POLICY
>
> So we have discovered an underlying model.
>
> Given an identity, we make a first-level attempt to
> authenticate it.  There are three possible outcomes:
> authentication fails, authentication passes, and status
> unknown.
>
> Given an authenticated identity, we make a second-level
> decision informed by reputation and accreditation systems.
> This decision is capable of both acceptance and rejection.
>
> (Given a non-authenticated identity, we may still choose to
> reject it; this gives the rejection orientation the benefit
> of the doubt.)
>
>
>                            * * *
>
>                     THE IDEALIZED LOGIC
>
> The Platonic version of the above logic goes something like
> this:
>
>  - Apply authentication test to $IDENTITY.
>  - If fail, REJECT.
>
>  - If unknown, apply rejection-oriented test to $IDENTITY.
>       - if accreditation/reputation fails, REJECT.
>       - else, proceed.
>
>  - If pass, apply acceptance AND rejection-oriented tests to $IDENTITY.
>       - if accreditation/reputation fails, REJECT.
>       - if accreditation/reputation passes, ACCEPT and skip further checks.
>       - else, proceed.
>
>  - proceed means "go to the next $IDENTITY".
>
> The first test is always authentication.  And when I say
> "authentication", I mean "does the sending entity
> represented by the $IDENTITY authorize the client to use its
> name".  Authentication is about the relationship between
> sender and publisher.  Simply: do they match?
>
> The second test is always policy.  Policy is a receiver-side
> thing: "does the receiving entity authorize the client to
> proceed."  Once sender and publisher have been confirmed to
> match, policy is about their relationship to the receiver.
>
> Note the similarity in structure between the authentication
> and the policy tests.  The similarity often means the two
> decision levels are conflated in practice.  But I will try
> to unravel them below.  I will outline 5 identities below;
> pretend you're a receiver, rotating through each identity in
> turn.
>
>                            * * *
>
>                 THE IDENTITY TESTS IN DETAIL
>
> Now is the time to be completist.  We'll look at six
> identities in total.
>
>
> 0: $IDENTITY = SMTP AUTH username
>
>    at the authentication level,
>     - fail means user unknown or bad password.
>     - pass means user is doing message submission.
>
>    at the policy level,
>     - fail might mean the user has gone over rate limits.
>     - pass means the user is allowed to submit mail to the
>       MSA, bypassing further checks.
>
>
> 1: $IDENTITY = IP address and PTR domain name.
>
>    at the authentication level,
>     - MTAMark/SS provide an example of a fail result:
>       "according to the network provider, this IP is not
>       authorized to act as a peer MTA on the internet."
>     - a pass under MTAMark/SS would mean "the network
>       provider requests that receivers allow the passage of
>       mail from this IP without let or hindrance."
>
>     - given that MTAMark/SS are not widely deployed, today
>       most IP auth tests as described here return "unknown".
>
>    at the policy level,
>     - fails come from DNS IP blacklists
>     - passes come from DNS IP whitelists,
>       or from "trusted networks" eg. an ISP must trust its
>       dialup netblock for submission.
>
>
> 2: $IDENTITY = HELO domain name.
>
>    at the authentication level,
>
>     - an SPF, DRIP, or CSV lookup might return a pass: "yes,
>       this HELO name may be used by the SMTP client."
>
>     - an SPF, DRIP, or CSV lookup might return a fail: "no,
>       the SMTP client is not allowed to use this HELO name."
>
>    at the policy level,
>
>     - passes and fails come from RHSBLs and RHSWLs.
>
>    See CSV for more detail: http://www.jlc.net/MARID/CSV/
>
>
> 3: $IDENTITY = return-path.
>
>    at the authentication level,
>
>     - RMX, DMP, and SPF all provide pass/fail/unknown
>       results.  SPF also returns softfail, etc.
>
>    at the policy level,
>
>     - passes and fails come from RHSBLs and local policy
>       engines.  In the case of SPF, accreditation can help
>       inform that decision.
>
>
> 4: $IDENTITY = header sender
>
>    at the authentication level,
>
>     - CallerID provides pass/fail/unknown
>     - SenderID provides pass/fail/unknown/softfail, etc.
>    
>    at the policy level,
>     - passes and fails come from RHSBLs and other richer
>       reputation systems and local policy engines.
>
>
> 5: $IDENTITY = cryptographic signature, usually matching
>                header "From:"
>
>    at the authentication level,
>
>     - does the crypto signature check out?
>       (In PGP, is the public key unrevoked on a keyserver?)
>       (In DK, is does the public key in DNS prove the signature?)
>
>     - also, does the crypto signer's identity match the
>       header "From:" (if it doesn't, it's easy for the MUA
>       to override the displayed "From:" with the signature's
>       identity.)
>
>    at the policy level,
>
>     - given that we've authenticated the content in the best
>       possible way, do we still want to read it?  Reputation
>       systems, accreditation systems, and local policy
>       engines can all key against the email address.
>
>
>                            * * *
>
>                            FUSION
>
> I see no reason why a sender might not want to publish
> records to help authenticate each of the identities shown
> above.
>
> I see no reason why a receiver would not want to check them
> all, too.
>
>
>                            * * *
>
>                 EXTRACTING THE COMMON FACTOR
>
> Each of the above identities contains at minimum a domain
> name; often you get a full email address with localpart.
>
> Accreditation and reputation systems find this convenient
> because they can key off domain names.
>
> Publishers find this convenient because they can publish a
> relatively small number of records in DNS; those records can
> be evaluated in different contexts but produce useful
> results in each case.
>
> Now, we have seen that SPF Classic and SenderID will both
> work with SPF TXT records.
>
> In fact, SPF Classic describes not just the return-path
> domain but the HELO domain as well.  So extending it to the
> PRA identity, as with SenderID, is a natural fit.  Extending
> it in the other direction, toward the PTR domain, works too.
>
> Also, we have seen that the SPF record can describe signed
> email independent of designated sender schemes:
>
>   v=spf1 dk -all
>
> So SPF can be made to work for all the identities described
> above!
>
>  - the PTR domain
>  - the HELO domain
>  - the return-path domain
>  - the PRA from the headers
>
>  - and the author, as represented by the "From" header
>    and/or the cryptographic signature.
>
> I propose that if the SPF record can be made to serve
> multiple identities, it is only logical to consider using it
> to provide as rich a set of semantics as possible.
>
>
>                            * * *
>
>    EXTENDING SPF: MATCHING SEMANTICS TO IDENTITY CONTEXT
>
> If an SPF record is evaluated in the context of a certain
> identity, the result of SPF evaluation should provide the
> appropriate semantics.
>
> In practice, this means:
>
>  1  When looked up against the PTR domain name, an SPF result
>     indicates whether a host should be an MTA or not.  If
>     so, policy decisions can be made against that domain
>     name.  If an IP address has no PTR record or if its PTR
>     domain name has no SPF record, the authorization result
>     is considered "unknown".
>
>     This use of SPF provides MTAMark/SS's essential semantics.
>
>  2  When looked up against the HELO domain name, an SPF
>     result indicates whether that domain name authorizes its
>     use by that mail server.  If so, policy decisions can be
>     made against that domain name.
>
>     This use of SPF provides CSV's essential semantics.
>
>  3  When looked up against the return-path domain name, an
>     SPF result indicates whether that domain name authorizes
>     its use by that mail server.  If so, policy decisions
>     can be made against that domain name.
>
>     This use of SPF provides SPF Classic's semantics.
>
>  4  SPF can be looked up against the Purported Responsible
>     Address from the headers.
>
>     This use of SPF provides CallerID's essential semantics
>     and is now called SenderID.
>
>  5  SPF can be looked up against the address in the header
>     "From:".  The discovered policies can be evaluated
>     whether they are designated-sender based or
>     cryptographic in nature.
>
>     This use of SPF helps provide a link to DomainKeys, PGP,
>     S/MIME, and so on.
>
>     Use of SPF against the header "From:" may intrigue
>     people; it deserves a little more explanation.
>
>     Suppose the header is "From: mengwong(_at_)yahoo(_dot_)com".
>
>     So we do the SPF lookup and find that the record there
>     is
>
>        yahoo.com  TXT  "v=spf1 dk -all"
>
>     That tells us: the message better be DK signed or it's
>     no good.  Nothing at all about whether it comes from a
>     Yahoo.com mail server IP address.  This is pure crypto.
>
>     We then apply the DK logic to the message to answer the
>     authentication test.
>
>     After the authentication test completes, we apply policy.
>
>     That's how SPF brings the header "From:" into the fold.
>
>
>                            * * *
>
> Now that we've taken a step back, I think we can resolve the
> sibling rivalry between the different proposals now on the
> table; after all, sibling rivalry means the proposals really
> are brother and sister.  The happy ending occurs when
> brother and sister realize they are family, and discover
> their roots.  The above argument presents a structure in
> which the proposals can stand together as a family.
>
> -------
> Sender Policy Framework: http://spf.pobox.com/
> Archives at http://archives.listbox.com/spf-discuss/current/
> Send us money!  http://spf.pobox.com/donations.html
> To unsubscribe, change your address, or temporarily deactivate your subscription, 
> please go to http://v2.listbox.com/member/?listname=spf-discuss(_at_)v2(_dot_)listbox(_dot_)com
--
Teddy's Computerworld      http://www.teddy.ch/
Himmelrainweg 2            mailto:teddy(_at_)teddy(_dot_)ch
4450 Sissach               076 383 80 60