From: Frank Ellermann
Sent: Friday, October 22, 2004 7:10 PM
Seth Goodman wrote:
In fact if someone is offering me a message, why should I
exclude myself from scrutinizing anything in that message
before deciding to accept it?
You are of course free to do with your mail whatever you like.
It's less simple if you are processing mail for 3rd parties
(= your users), because then you can't do whatever you want
without prior consent. Hector often cited some of the legal
aspects here. E.g. GMail might be illegal where I live, and
maybe also in California.
There are serious discussions in German net-abuse groups about
adding the word ***Spam*** to the subject. Some experts think
that this might be illegal. Fortunately my provider ignored
these discussions and does it anyway.
I can understand people being nervous about modifying content in the message
body. The SPF header, however, should be considered as trace header
information. Even though in the strictest sense, everything past the start
of data is content, (2)822 clearly expects hosts to add trace header
information to the message headers and virtually everyone does.
Perhaps we are misunderstanding each other (how incredibly unusual for this
list!). I as not suggesting changing anything, beyond the MSA, and it is
not even necessary to change anything there, if the MSA is willing to reject
instead. What I was suggesting included two things:
1) MSA's SHOULD enforce submission rights. I really wish the RFC's had put
some teeth into this and said MUST, but they had their own constraints when
the standards were drafted and we have to deal with what we've got. MSA's
still can enforce submission rights in several ways.
a) MSA immediately rejects, with explantation, any message where the highest
of From:/Sender:/(newest)Resent-From: does not match return-path. This
would be ideal, though I don't know if all MUA's can set return-path
properly. This approach avoids all ethical and legal problems. If the
explanation returned included web links to clear instructions on how to
correct the setup in each of the common MUA's, service calls would be
reduced, but not eliminated.
b) MSA enforces equivalence by changing/adding headers according to the
following rules: if return-path is different from the authenticated
identity or missing, set return-path to the authenticated identity; if From:
is different from the authenticated identity and there is no sender, add
Sender: with the authenticated identity; if Sender: is different from the
authenticated identity and there is no Resent-From:, change Sender: to
Original-Sender: and add Sender: with the authenticated identity; if newest
Resent-From: is different from the authenticated identity, change
Resent-From to Original-Resent-From: and add Resent-From: with the
authenticated identity. This approach will generate few if any service
calls (only very knowledgeable users would even notice this), but is perhaps
on shaky legal footing. Perhaps one of the attorney's on the list could
comment? I wouldn't like my mail tampered with this way, but I'm not an
average user.
c) MSA's for hotel rooms/greeting card sites are in a somewhat different
position. They have no ability to validate that the user has the right to
use any particular address. A reasonable approach for them is to act the
same way as a mailing list: preserve the From:, add their own address both
for Sender: and return-path. The site sending the message has to take
responsibility for it. Just as what mailing lists should but often don't
do, if there is already a Sender: or Resent-From: header, the MSA should
change that to Original-Sender: or Original-Resent-From: and put their own
address in Sender: or Resent-From:.
2) Recipients would read the sender's SPF record, and if there is a
2821=2822 header equivalence statement, would check the 2822 headers for
equivalence during the SMTP transaction. If they did not match, the
recipient could do one of two things, depending on local policy:
a) Reject the message at the end of data with an appropriate 5xx error
message. This is respecting the sender's anti-forgery policy.
b) If the message is otherwise acceptable, add an SPF header that includes
the fact that the 2821 and 2822 headers should have agreed, according to the
sender, but didn't. In this case, it's up to the MUA and user to sort it
out, which is why I don't like this solution. Since the SPF header is a
trace header, I see no ethical or legal issues with it.
the world is moving in the direction of requiring the at
least the 2821 and 2822 domains to be the same. Eventually,
it may get more strict.
Some years ago the world was apparently moving in the direction
of using PGP or S/MIME everywhere. Let's see what happens.
I can't argue with the failure of that prediction, but I didn't make that
one :) The reason for the world moving in the direction of 2821 and 2822
identities being the same is that there is no other relatively simple way to
stop this important type of forgery. I've seen proposals that each of the
identities would be validated separately. That is low overhead only if the
validation mechanism is designated sending IP, and that only works on the
first hop. Though people can dismiss forwarding as a minor part of mail
traffic, if the authentication means for these various identities work on
the first hop only, you can be sure that spam will move for mock forwarding
to get around it.
For an authentication mechanism to be worth anything, it has to work
end-to-end. In that scenario, it does not make sense to validate the
identities separately, unless each includes strong crypto that signs message
content to avoid replay. That puts an unacceptably high burden on
recipients, IMHO. As a recipient, I don't want to validate six different
public key signatures before I can reject a forgery.
Let's go back to basics. An MUA authenticates itself to an MSA and then
submits a message to the MSA. The MSA has an authenticated identity for the
MUA and a message submitted by the MUA. That is the obvious place to
prevent forgery, using whatever method is ethical/legal for the MSA. This
will vary from locale to locale, but the bottom line is that the MSA is in
the best position to prevent forgery since the MUA has to authenticate to
it. So what constitutes forgery? Perhaps this is the key to the discussion
and where people might disagree. Here's my definition: using an identity
that the MUA does not have the right to use.
That doesn't mean that the identities have to be the same, but that the MSA
has to be able to ascertain that the MUA has the rights to any identity that
it claims. Obviously, the MSA knows that the MUA has the rights to use the
identity that authenticated with. Beyond that, it gets complicated. There
is no technical reason why we couldn't build a system where the MSA can
verify the MUA's right to use alternate identities. There are a lot of ways
to do that: certificates, direct communication with MX's for the foreign
domain, records in DNS, etc. There is nothing technically wrong with this
concept, except that no ISP is going to go to this trouble. Absent such a
system, the only way for an MSA to prevent forgery is to limit the sending
identities, both 2821 and 2822, to the identity that the MUA authenticated
with. Though this is not optimal and not completely general, it is the
simplest way to solve the problem while inconveniencing the minimum number
of clients. Since it is opt-in, in doesn't break anything that doesn't
participate. However, I do predict, for all of the reasons above, that this
will gradually become the expected practice. Perhaps a system that allows
MSA to validate an MUA's right to use a foreign address will come to pass.
That would be optimal, but I doubt that will happen. Requiring equivalence
is a 90% solution to the problem that requires a fraction of the effort of
the 100% solution, so that is the basis for my prediction that the two
identities will _eventually_ be expected to match for good delivery rates.
This implies to me that is SHOULD be the sender. That
doesn't ay we have to enforce it, but it also doesn't say
that we _can't_ enforce it, especially if the sender agrees.
The most simple approach would be, that those who "want" a
Sender header for whatever reasons (incl. PRA and "eh") simply
take the Return-Path if necessary. But we had this discussion
already, it's IMO the technical reason why Sender-ID and MARID
failed. With a Return-Path default Sender-ID could work.
I agree that this approach is not workable. You can't assume a header that
is not present. It is also very questionable for a recipient to add any
originator headers to a message. I am suggesting that the header is either
there or it isn't, and recipients evaluate the message using the sender
policy that tells them if the sender enforces the equivalence, and thus has
the appropriate headers, or they don't.
what are the real benefits of continuing to allow the two
addresses to be from different domains?
The same reason why you always use the same postal address in
your snail mails no matter where you "submit" them.
That's not a comparable analogy. All postal boxes are equivalent, both
legally and practically speaking. In the postal mail system, the sender is
in legal trouble if the envelope return address is fraudulent. Thus for
postal mail, the submitting location is irrelevant, since there are adequate
laws to protect recipients. Because of these protections, postal boxes can
be placed anywhere and anyone can drop mail into it.
Unfortunately, the email system was architected completely differently and
was not supported by any legal framework until very recently. There is no
legal requirement that the MAIL FROM: address be correct, or even that it
really exists. Perhaps the U.S. CAN-SPAM ACT does actually require this,
but it is nothing but a toothless tiger. Thus, recipients have been forced
to defend their own interests by creating blacklists of senders, by sending
IP, known to send out fraudulently addressed and unsolicited mail.
Fortunately, the email system runs on TCP/IP, which makes it very difficult
to forge am IP address and conduct a full TCP transaction. There is no
equivalent in the postal mail case. The closest thing is the postmark
imprinted by the post office, but that represents hundreds or thousands of
individual postal boxes, so there is no effective identification who put the
mail into the postal box, nor no real need for it.
Let's explore this further, please.
No further ideas, maybe news2mail gateways: If you post an
article somewhere using your normal From address, and if the
newsgroup is exported to a mailing list somewhere else, then
the mail has your From-address, and a MAIL FROM gateway. In
that case it depends on the gateway, maybe it adds the same
address as Sender: gateway hiding your Sender (if any) as a
X-Original-Sender (or similar), or it uses Errors-To like a
SYMPA mailing list keeping your From / Sender as is.
AFAIK gmane does "the right thing" from your POV, please check
it in the header of this mail. I add a dummy Sender: matching
my From: to make it more interesting for gmane and listbox ;-)
If you don't see any "Test for Seth" in the headers this test
failed miserably, and either Gmane or Listbox deleted my Sender.
Looks like it didn't quite make it through, but I think that is because of
normal mailing list behavior. I suspect that the mailing list software took
the sending identity as From: and ignored your Sender:. You can see the
headers on the post as received by you, so I won't post them here.
What I meant to say was the fact that the sender asserted
2821=2822 equivalence and the message passed that test. This
would have to be communicated in the SPF header
Yes. It's possible to split the work in other ways between MTA
and MUA, e.g. the MUA could check the equivalence, and the MTA
only tests 2821 (adding an "eh" info for the MUA).
That's certainly possible, though not desirable, IHMO. The problem is we
want to reject forgeries, not silently discard them. Since MUA's should
never attempt to concoct a "bounce", there is no other choice but to
silently discard. When a domain publishes the fact that it _always_
enforces equivalence between 2821 and 2822 headers, it is saying "please
reject any message claiming to come from us where the 2821 and 2822
identities are not the same because they are forgeries".
pobox.com users have per user policies, that's as good as a
real domain for SPF. So maybe Meng sees another good reason
to support William's idea.
I don't think it's fair to criticize Meng for that.
Oops, where do you see a criticism here ? It's a GOOD feature
to offer per-user policies. And with per-user policies you can
solve many odd cases to get the same sender policy working with
both 2821 and 2822 tests.
My misinterpretation, sorry.
--
Seth Goodman