From: Frank Ellermann [mailto:nobody(_at_)xyzzy(_dot_)claranet(_dot_)de]
Sent: Sunday, August 14, 2005 6:40 PM
Seth Goodman wrote:
there is no MUST that I can find that says it has to have any
relation to the actual sender
Yes, that used to be obvious, and the 2119 keywords like MUST
were invented after 821 / 1123.
RFC2821 and RFC2822 do define and use those keywords (see RFC2821, section
2.3 and RFC2822, section 1.2.1). Together, they replace 821, 822, 974 and
1869 plus update and clarify 1035 and 1123.
If an RFC defines X = Y, then it cannot say "<X> MUST be <Y>",
it's a definition and no requirement. So the MAIL FROM is the
Return-Path, it's not any "report-trouble-elsewhere-address".
I'm afraid that's exactly how it is described, even though I dislike that
concept at least as much as you. Here are some citations to support this
view:
----------------------------
[RFC2821]
2.3.1 Mail Objects
SMTP transports a mail object. A mail object contains an envelope
and content.
The SMTP envelope is sent as a series of SMTP protocol units
(described in section 3). It consists of an originator address (to
which error reports should be directed); one or more recipient
addresses; and optional protocol extension material.
----------------------------
[RFC2821, section 3.3, Mail Transactions]
The first step in the procedure is the MAIL command.
MAIL FROM:<reverse-path> [SP <mail-parameters> ] <CRLF>
This command tells the SMTP-receiver that a new mail transaction is
starting and to reset all its state tables and buffers, including any
recipients or mail data. The <reverse-path> portion of the first or
only argument contains the source mailbox (between "<" and ">"
brackets), which can be used to report errors (see section 4.2 for a
discussion of error reporting).
----------------------------
[RFC2821, section 3.7, Relaying]
If an SMTP server has accepted the task of relaying the mail and
later finds that the destination is incorrect or that the mail cannot
be delivered for some other reason, then it MUST construct an
"undeliverable mail" notification message and send it to the
originator of the undeliverable mail (as indicated by the reverse-
path).
----------------------------
[RFC2821]
3.8.5 Envelopes in Gatewaying
Similarly, when forwarding a message from another environment into
the Internet, the gateway SHOULD set the envelope return path in
accordance with an error message return address, if supplied by the
foreign environment. If the foreign environment has no equivalent
concept, the gateway must select and use a best approximation, with
the message originator's address as the default of last resort.
----------------------------
[RFC2821, section 4.1.1.2 MAIL (MAIL]
The reverse-path consists of the sender mailbox. Historically, that
mailbox might optionally have been preceded by a list of hosts, but
that behavior is now deprecated (see appendix C). In some types of
reporting messages for which a reply is likely to cause a mail loop
(for example, mail delivery and nondelivery notifications), the
reverse-path may be null (see section 3.7).
----------------------------
[RFC2821, section 4.4, Trace Information]
It is possible for the mailbox in the return path to be different
from the actual sender's mailbox, for example, if error responses are
to be delivered to a special error handling mailbox rather than to
the message sender. When mailing lists are involved, this
arrangement is common and useful as a means of directing errors to
the list maintainer rather than the message originator.
<...>
The primary purpose of the Return-path is to designate the address to
which messages indicating non-delivery or other mail system failures
are to be sent. For this to be unambiguous, exactly one return path
SHOULD be present when the message is delivered. Systems using RFC
822 syntax with non-SMTP transports SHOULD designate an unambiguous
address, associated with the transport envelope, to which error
reports (e.g., non-delivery messages) should be sent.
Historical note: Text in RFC 822 that appears to contradict the use
of the Return-path header (or the envelope reverse path address from
the MAIL command) as the destination for error messages is not
applicable on the Internet. The reverse path address (as copied into
the Return-path) MUST be used as the target of any mail containing
delivery error messages.
Of course you can twist it to get this effect, and in many
cases like mailing-lists automatically unsubscribing bouncing
addresses that makes sense.
The last citation above makes it clear that this is not a twisted
interpretation. While I personally believe that it is twisted concept to
have anybody _but_ the originator get bounces for messages they sent, with
the obvious exception of mailing lists, the standard clearly allows that.
While one could make a rather weak argument for some other local mailbox at
a domain getting bounces other than the sender, I think we can safely say
that there is never a valid reason for the domain to be different in the
return-path and From: headers. If bounces really did need to go to another
domain, for that one-in-a-million corner case, they can always forward it to
accomplish the goal.
In other cases it makes no sense, if mail from A does not make
it to B, then the very last you need to analyze this problem
between A and B is a third system MAIL FROM C.
Agreed. This is totally screwed up, but permitted.
But MAIL FROM troubleshooter(_at_)A instead of stupid_user(_at_)A can be
a good idea. That's a variant of what mailing lists do, they
don't use MAIL FROM list_member(_at_)elsewhere, they also don't use
MAIL FROM list_owner(_at_)here, they probably use
list_bounces(_at_)here(_dot_)
Yes, this is one of the only possible use cases where it makes sense for the
higher of From:, Sender:, Resent-From: or Resent-Sender: to be different
from the return-path. But really, have you ever worked in a place where
they actually had someone designated to receive bounces for you? Would you
even want that? For a human mail sender, I think this is a theoretical idea
but not a real case. Even the nonsense about secretaries sending mail on
behalf of their bosses is fanciful and a throwback to snail mail that should
have been removed. While I used to have a secretary who did exactly this in
the snail mail days, since the advent of email and word processors, I have
never had anyone to do those things for me despite holding "better"
positions than when I had a secretary. Not to mention the fact the great
majority of MUA's out there don't even support this feature, nor do they
support the feature of multiple From: addresses.
[arbitrary "errors-to"]
This is obviously different from today's best commercial
practice, and, in fact, the operating assumption behind SPF.
It's also different from the original 821 concept. The special
case "user not local" was always a special case. BTW, that's
IMHO all clear since at least 2003 and RMX, anything new ?
I thought RMX was a great idea and supported it, but it got zero adoption
and is all but off the radar today. The "user not local" may be a special
case, but virtually none of the ISP's out there stop you from using whatever
return-path you want, local or not. While it is theoretically a special
case, ISP's allowing this every day elevates the special case in the
standard to the general case in practice, though I completely disagree with
it.
Though the following predates your 2003 marker by two years,
----------------------------
[RFC2821, section 7.1, Mail Security and Spoofing]
Efforts to make it more difficult for users to set envelope return
path and header "From" fields to point to valid addresses other than
their own are largely misguided: they frustrate legitimate
applications in which mail is sent by one user on behalf of another
or in which error (or normal) replies should be directed to a special
address. (Systems that provide convenient ways for users to alter
these fields on a per-message basis should attempt to establish a
primary and permanent mailbox address for the user so that Sender
fields within the message data can be generated sensibly.)
----------------------------
Needless to say, I think the above is misguided. People should not forge
identities, even if the use is benign.
<...>
[outgoing SPF]
SPF happens to be a great tool for that, even for ISP's that
don't publish their own SPF record or check SPF on incoming.
Julian has implemented this. I'm more a fan of the "enforced
submission rights" in 2476(bis), but it's a similar direction.
I completely agree. The originating MTA has to take responsibility for the
traffic it injects into the mail stream. Without enforcing submission
rights, it can only do a mediocre job, at best.
Mailing lists are classified as remailers and have to keep
the From: and add their own address as Sender:.
The latter is AFAIK nowhere required. Many mailing lists do
it, but not all. IMHO they could restrict themselves to the
standard List-header fields and not touch any existing Sender.
You're technically correct, they don't have to add a Sender: header. What
the standards say is that as remailers, they MUST change the return-path and
they SHOULD add some other header to show the remailing event. Sorry for
the long quote, but it is relevant.
----------------------------
[RFC2822, section 3.6.6, Resent fields]
3.6.6. Resent fields
Resent fields SHOULD be added to any message that is reintroduced by
a user into the transport system. A separate set of resent fields
SHOULD be added each time this is done. All of the resent fields
corresponding to a particular resending of the message SHOULD be
together. Each new set of resent fields is prepended to the message;
that is, the most recent set of resent fields appear earlier in the
message. No other fields in the message are changed when resent
fields are added.
<...>
Resent fields are used to identify a message as having been
reintroduced into the transport system by a user. The purpose of
using resent fields is to have the message appear to the final
recipient as if it were sent directly by the original sender, with
all of the original fields remaining the same. Each set of resent
fields correspond to a particular resending event. That is, if a
message is resent multiple times, each set of resent fields gives
identifying information for each individual time. Resent fields are
strictly informational. They MUST NOT be used in the normal
processing of replies or other such automatic actions on messages.
Note: Reintroducing a message into the transport system and using
resent fields is a different operation from "forwarding".
"Forwarding" has two meanings: One sense of forwarding is that a mail
reading program can be told by a user to forward a copy of a message
to another person, making the forwarded message the body of the new
message. A forwarded message in this sense does not appear to have
come from the original sender, but is an entirely new message from
the forwarder of the message. On the other hand, forwarding is also
used to mean when a mail transport program gets a message and
forwards it on to a different destination for final delivery. Resent
header fields are not intended for use with either type of
forwarding.
----------------------------
According to this standard, mailing lists should use Resent-From:, but the
de facto practice has been to use Sender:. No one else seemed to ever use
Sender:, anyway, and it does indicate sending mail on someone else's behalf.
What is not OK to do is just change the return-path to your own and change
the To: header. While I don't care for SID and care even less for the
technically challenged organization responsible for it, remailers SHOULD
append another header, as should anyone else sending a message on behalf of
the original author.
The last sentence in the above citation makes it perfectly clear that the
requirement in SID for forwarders to add Resent-*: headers is contrary to
the RFC's, not to mention actual practice since the beginning of email.
Systems that do not comply with this are BROKEN.
IBTD. Let the "Sender" be what it always was, a more or less
irrelevant feature if sender != author, or if there are two or
more authors.
I'm OK with that, as it never happens anyway. Remailers SHOULD then use
Resent-From: instead. My point is that there is no reason to allow anyone
to change the return-path on a message without updating the headers to show
what happened.
MS MUA's, which unfortunately are the predominant ones on
the planet, only support what I would call encapsulation
forwarding. In this style, it is treated as a new message
with appropriate headers for the new sender.
Actually that's fine if they only would use a message/rfc822
or multipart/digest as specified for minimal MIME conformance.
That's the least of it. Their message store is not 822-compliant and they
routinely destroy all MIME armor, making subsequent interpretation of
messages by other applications extremely challenging. I guess they have to
answer for their design choices.
The forwarded message is either in-line with the body or as
a MIME attachment
Forwarded mails are not necessarily a "MIME attachent", the
default Content-Disposition is _inline_ for message/rfc822.
For obscure reasons MS twists that into _attachment_ with a
type "eml", but that is their own proprietary invention, it
is legal but unnecessary. Actually it's a royal PITA.
For everybody. You'd think that by 2005 they would have worked their way
over to the standards.
Thunderbird is similar to MS MUA's in this regard, as I
believe are Netscape and Mozilla Suite.
My vintage 1997 "Mozilla 3" uses the default, no attachment.
They didn't have a vested interest in being incompatible with everything
else.
Pine handles the "redirect" style of forwarding properly
For a definition of "proper = Resent". MIME introduced the
more flexible solutions later. 2822 twisted the old Resent-
style into "multiple blocks of Resent-headers", but that's
FUBAR. Just look into the Sender-ID draft, it's a major pain
to "decode" this correctly at the receiver.
I agree, it's not great. OTOH, virtually nobody uses it as originally
intended in 822/2822, except for a few Pine users and perhaps some other
minor MUA's. FWIW, I disagree with the philosophy of this style of
forwarding, and it wouldn't break my heart to see it disappear. It
virtually has by lack of adoption. The rationale behind it is to allow a
recipient to forward a message while making it appear the message came from
the original author.
While there a few theoretical corner cases where this might be a
convenience, IMHO, the downside far outweighs the upside. If the new
recipient happens to notice, they will see it didn't come from the original
author. Of course, they may _not_ notice that, and the result is then very
close to forgery. If you want to forward a message to someone, then just
inline forward it. Forget about trying to fool the recipient into thinking
it came from the original sender. What's the big deal in clearly revealing
your identity and taking responsibility for a message that _you_ sent? Is
there some real case where a role account owner _can't_ deal with an inline
(or attachment) forwarded message?
<...>
I would argue that should we eventually decide to apply the
SPF record to the current sender in the 2822 headers, it
would not be that hard to fix the few problems that would
result.
It's impossible, the cases without Sender and From != MAIL FROM
don't go away by wishful thinking. That's the kind of wishful
thinking you find in the Sender-ID drafts.
Please don't say "you" and "Sender-ID drafts" in the same sentence, thank
you very much. Clearly, MAIL FROM != From (or higher originator headers)
some of the time today. That is why I agree that it is impractical to do
any kind of SPF checking on addresses in the message body, _today_.
However, SPF does encourage tightening up submission rights, and SPF does
appear to be gaining ground. What does that mean? For a start, it means
that more sites will control the return-path that people claim. Even though
the RFC's appear to forbid this next step, it is not that much of a stretch
to start requiring the highest originator header to match the return-path
domain (at least). If you want to stop outgoing message forgery, that is
where you ultimately have to go.
The most important question you asked, "Isn't this just wishful thinking?",
deserves a serious answer. I suggest that as SPF gains deployment, MTA's
that send a lot of messages that fail SPF are going to get bad reputations.
This will cause them difficulty delivering legitimate mail. The same will
ultimately happen if they deliver a lot of spam that does pass SPF. If
neither of those happens, I would say that SPF would be a failure. However,
I don't see it playing out that way.
One way for ISP's to partially avoid this is to do an SPF check on outgoing
mail. However, that doesn't address the root cause, which is uncontrolled
submission rights. If they want to keep their reputation up, they will
ultimately have to enforce submission rights more than they do today. That
means some means of controlling use of foreign domain names, and in the
headers as well as the return-path. Think about it: how many _real_
senders, not concocted corner cases, would be adversely affected if their
ISP required the return-path to match the highest originator header _and_
the resulting message to pass an outgoing SPF check? This doesn't limit the
domains you can use. It just says that the 2822 headers have to reasonably
match the 2821 envelope and SPF is the arbiter of whether or not you can use
the domain name. Since the SPF record is published by the domain owner,
this appears to be a legitimate restriction.
It's also the reason why SPF and Sender-ID have an impressive
IESG note which is essentially the polite form of "FUBAR". Bye
I think the well-deserved FUBAR derives largely from their attempt to
repurpose Resent-*: contrary to 20+ years of standards and commercial
practice. Fortunately, we made their case even weaker by proposing the
SUBMITTER SMTP extension. Had those two technical errors not occurred, I
think the technical concept would have had some merit. FWIW, I'm glad they
screwed up technically, as in retrospect, their strategy amounted to, "We're
either going to own this thing or kill it". When it became clear their
proposal was about as popular as the bubonic plague, they opted for the
latter. Why the IETF, as a technical organization, will even give them the
time of day after that debacle is a complete mystery to me.
--
Seth Goodman