I have a funny feeling that MARID will eventually engender a
fusion of the proposals that are now on the table. And when
I say "fusion", I mean the same kind of "fusion" that puts
on a single plate spicy spring rolls with creme fraiche and
smoked salmon drizzled with a curry Bearnaise sauce.
But that kind of fusion is ultimately okay. Maybe it'll
taste good!
I expect the same to happen with the work we are doing.
Let me explain.
* * *
FOUR IDENTITIES
So far we've seen four schools of channel-based
authentication, each focusing on a different identity:
- the IP address and its related PTR record
- the HELO domain
- the return-path
- the Purported Responsible Address from the headers
It doesn't have to be winner-takes-all. I think sensible
people will find a way to get useful work out of each
identity.
* * *
ACCEPTANCE AND REJECTION
There are acceptance-oriented and rejection-oriented ways to
scrutinize incoming email.
An acceptance orientation lets a receiver say "oh, I
recognize the sender; they're trusted. I'll bypass further
checks."
A rejection orientation lets a receiver say, "hm, there's
something fishy about this; I might reject them, or I might
subject them to a biased scrutiny."
We have seen, to date, that the designated sender components
of SPF make it possible to perform channel-based tests on
the three domain-related identities above. But the overall
framework of the SPF syntax makes it possible to do
content-based tests as well --- you can define "pgp",
"smime", "dk" and such mechanisms. Just keep that in mind;
it'll be useful later.
* * *
A brief note about SPF and SenderID.
I support SenderID and believe it will provide value.
However, I have chosen to use the name "SPF" in this email
in a lot of places. This is for technical reasons.
When I use the term "SPF", I am referring to the function
that returns an authentication result based on a tuple of IP
address and domain name. (This is for DS. For DK, it
provides a link, and other code has to handle the crypto.)
When this function is applied to the return-path, we have
"SPF Classic"; when it is applied to the PRA from the
headers, it is "SenderID". When it is applied to other
identities, it becomes other things. More on that later.
* * *
WHAT THE PROPOSALS WANT SENDERS TO DO
Each proposal chooses an identity and asks sending sites to
slap new DNS records on that chosen identity.
In MTAMark and SS type proposals, the records go on the IP
address's PTR.
In DRIP and in Dave Crocker's CSV proposal, the records go
onto the HELO domain name.
In SPF Classic, the records go on the return-path domain
name and the HELO domains too.
In CallerID and SenderID, the records go on the PRA
domain.
In DomainKeys, the records contain public keys, and work
for the "From:" identity.
* * *
SUPPOSE WE DO ALL OF THE ABOVE
Let's suppose a well-respected sender decides to play it
safe and do all of the above. It slaps SPF records on the
HELO domain and the return-path domain. It applies to
trusted-forwarder.org to get its IP addresses listed. It
very carefully prepends the right header in mail so the PRA
algorithm returns the right result. It even signs its
messages using DomainKeys. Let's further suppose that
sender has acquired accreditation credentials, and enjoys a
good reputation in third-party reputation systems also. So
it's got all its bases covered.
When a receiver sees a connection coming from that sender,
it has a surfeit of identities it can use.
It could look up the IP address in trusted-forwarder.org.
If the IP address is found, the receiver could say "okay,
it's trusted; we'll whitelist it based on IP."
But if the IP wasn't found, the receiver could look up the
HELO name that identifies the sending server. The receiver
performs SPF checks on the helo domain and confirms that the
use of the name is authorized by that name. If the name is
thereby authenticated, and if the name is known (via some
external lookup into an accreditation or reputation system)
to be trusted, then the acceptance orientation kicks in, and
the receiver says "okay, it's trusted; we'll whitelist it
based on who the server is." I'm thinking of taxis lined up
at the airport, all trusting that they'll get a good fare
simply because the customer's walking out of a certain door.
If the HELO server isn't trusted, the receiver could look up
the identity in the return-path. Check the SPF on that, and
again the receiver gets to play the acceptance / rejection
game.
Next the header "Sender" (and others from CallerID).
Finally the "From" if the message was cryptographically
signed.
Sensible senders will want to make life easier for receivers
by covering all their bases. Sensible receivers will use
all the inputs available to make the best decision.
This is an important principle we should keep in mind.
* * *
TWO LAYERS OF TESTS: AUTHENTICATION AND POLICY
So we have discovered an underlying model.
Given an identity, we make a first-level attempt to
authenticate it. There are three possible outcomes:
authentication fails, authentication passes, and status
unknown.
Given an authenticated identity, we make a second-level
decision informed by reputation and accreditation systems.
This decision is capable of both acceptance and rejection.
(Given a non-authenticated identity, we may still choose to
reject it; this gives the rejection orientation the benefit
of the doubt.)
* * *
THE IDEALIZED LOGIC
The Platonic version of the above logic goes something like
this:
- Apply authentication test to $IDENTITY.
- If fail, REJECT.
- If unknown, apply rejection-oriented test to $IDENTITY.
- if accreditation/reputation fails, REJECT.
- else, proceed.
- If pass, apply acceptance AND rejection-oriented tests to $IDENTITY.
- if accreditation/reputation fails, REJECT.
- if accreditation/reputation passes, ACCEPT and skip further checks.
- else, proceed.
- proceed means "go to the next $IDENTITY".
The first test is always authentication. And when I say
"authentication", I mean "does the sending entity
represented by the $IDENTITY authorize the client to use its
name". Authentication is about the relationship between
sender and publisher. Simply: do they match?
The second test is always policy. Policy is a receiver-side
thing: "does the receiving entity authorize the client to
proceed." Once sender and publisher have been confirmed to
match, policy is about their relationship to the receiver.
Note the similarity in structure between the authentication
and the policy tests. The similarity often means the two
decision levels are conflated in practice. But I will try
to unravel them below. I will outline 5 identities below;
pretend you're a receiver, rotating through each identity in
turn.
* * *
THE IDENTITY TESTS IN DETAIL
Now is the time to be completist. We'll look at six
identities in total.
0: $IDENTITY = SMTP AUTH username
at the authentication level,
- fail means user unknown or bad password.
- pass means user is doing message submission.
at the policy level,
- fail might mean the user has gone over rate limits.
- pass means the user is allowed to submit mail to the
MSA, bypassing further checks.
1: $IDENTITY = IP address and PTR domain name.
at the authentication level,
- MTAMark/SS provide an example of a fail result:
"according to the network provider, this IP is not
authorized to act as a peer MTA on the internet."
- a pass under MTAMark/SS would mean "the network
provider requests that receivers allow the passage of
mail from this IP without let or hindrance."
- given that MTAMark/SS are not widely deployed, today
most IP auth tests as described here return "unknown".
at the policy level,
- fails come from DNS IP blacklists
- passes come from DNS IP whitelists,
or from "trusted networks" eg. an ISP must trust its
dialup netblock for submission.
2: $IDENTITY = HELO domain name.
at the authentication level,
- an SPF, DRIP, or CSV lookup might return a pass: "yes,
this HELO name may be used by the SMTP client."
- an SPF, DRIP, or CSV lookup might return a fail: "no,
the SMTP client is not allowed to use this HELO name."
at the policy level,
- passes and fails come from RHSBLs and RHSWLs.
See CSV for more detail: http://www.jlc.net/MARID/CSV/
3: $IDENTITY = return-path.
at the authentication level,
- RMX, DMP, and SPF all provide pass/fail/unknown
results. SPF also returns softfail, etc.
at the policy level,
- passes and fails come from RHSBLs and local policy
engines. In the case of SPF, accreditation can help
inform that decision.
4: $IDENTITY = header sender
at the authentication level,
- CallerID provides pass/fail/unknown
- SenderID provides pass/fail/unknown/softfail, etc.
at the policy level,
- passes and fails come from RHSBLs and other richer
reputation systems and local policy engines.
5: $IDENTITY = cryptographic signature, usually matching
header "From:"
at the authentication level,
- does the crypto signature check out?
(In PGP, is the public key unrevoked on a keyserver?)
(In DK, is does the public key in DNS prove the signature?)
- also, does the crypto signer's identity match the
header "From:" (if it doesn't, it's easy for the MUA
to override the displayed "From:" with the signature's
identity.)
at the policy level,
- given that we've authenticated the content in the best
possible way, do we still want to read it? Reputation
systems, accreditation systems, and local policy
engines can all key against the email address.
* * *
FUSION
I see no reason why a sender might not want to publish
records to help authenticate each of the identities shown
above.
I see no reason why a receiver would not want to check them
all, too.
* * *
EXTRACTING THE COMMON FACTOR
Each of the above identities contains at minimum a domain
name; often you get a full email address with localpart.
Accreditation and reputation systems find this convenient
because they can key off domain names.
Publishers find this convenient because they can publish a
relatively small number of records in DNS; those records can
be evaluated in different contexts but produce useful
results in each case.
Now, we have seen that SPF Classic and SenderID will both
work with SPF TXT records.
In fact, SPF Classic describes not just the return-path
domain but the HELO domain as well. So extending it to the
PRA identity, as with SenderID, is a natural fit. Extending
it in the other direction, toward the PTR domain, works too.
Also, we have seen that the SPF record can describe signed
email independent of designated sender schemes:
v=spf1 dk -all
So SPF can be made to work for all the identities described
above!
- the PTR domain
- the HELO domain
- the return-path domain
- the PRA from the headers
- and the author, as represented by the "From" header
and/or the cryptographic signature.
I propose that if the SPF record can be made to serve
multiple identities, it is only logical to consider using it
to provide as rich a set of semantics as possible.
* * *
EXTENDING SPF: MATCHING SEMANTICS TO IDENTITY CONTEXT
If an SPF record is evaluated in the context of a certain
identity, the result of SPF evaluation should provide the
appropriate semantics.
In practice, this means:
1 When looked up against the PTR domain name, an SPF result
indicates whether a host should be an MTA or not. If
so, policy decisions can be made against that domain
name. If an IP address has no PTR record or if its PTR
domain name has no SPF record, the authorization result
is considered "unknown".
This use of SPF provides MTAMark/SS's essential semantics.
2 When looked up against the HELO domain name, an SPF
result indicates whether that domain name authorizes its
use by that mail server. If so, policy decisions can be
made against that domain name.
This use of SPF provides CSV's essential semantics.
3 When looked up against the return-path domain name, an
SPF result indicates whether that domain name authorizes
its use by that mail server. If so, policy decisions
can be made against that domain name.
This use of SPF provides SPF Classic's semantics.
4 SPF can be looked up against the Purported Responsible
Address from the headers.
This use of SPF provides CallerID's essential semantics
and is now called SenderID.
5 SPF can be looked up against the address in the header
"From:". The discovered policies can be evaluated
whether they are designated-sender based or
cryptographic in nature.
This use of SPF helps provide a link to DomainKeys, PGP,
S/MIME, and so on.
Use of SPF against the header "From:" may intrigue
people; it deserves a little more explanation.
Suppose the header is "From: mengwong(_at_)yahoo(_dot_)com".
So we do the SPF lookup and find that the record there
is
yahoo.com TXT "v=spf1 dk -all"
That tells us: the message better be DK signed or it's
no good. Nothing at all about whether it comes from a
Yahoo.com mail server IP address. This is pure crypto.
We then apply the DK logic to the message to answer the
authentication test.
After the authentication test completes, we apply policy.
That's how SPF brings the header "From:" into the fold.
* * *
Now that we've taken a step back, I think we can resolve the
sibling rivalry between the different proposals now on the
table; after all, sibling rivalry means the proposals really
are brother and sister. The happy ending occurs when
brother and sister realize they are family, and discover
their roots. The above argument presents a structure in
which the proposals can stand together as a family.