Hi John,
Good discussion. I appreciate your time and effort on this. I'll try to avoid
sloppy statements that generate unnecessary discussion.
At 09:47 AM 1/24/2009 -0500, John C Klensin wrote:
--On Saturday, January 24, 2009 6:23 -0700 David MacQuigg wrote:
...
I have yet to see any sensible use-case that would be
complicated by a requirement that the HELO/EHLO name end in a
registered, verifiable domain name.
...
David,
Today, under 5321, EHLO clearly requires a fully-qualified
domain name that can be resolved unless the host doesn't know
its name and has to use an IP address, a situation that the
spec doesn't encourage. My reading of 5321 (but not
necessarily 821) says that HELO does too.
Perhaps it is just a matter of emphasis. As I read the spec
it *does* seem to encourage using HELO/EHLO names that are
useless for authentication. For example, the language in
4.1.4 "if the verification fails, the server MUST NOT refuse
to accept a message" puts the burden on the receiver to deal
with mis-configured transmitters. Receivers are expected to
accept mail from transmitters that say "HELO this is Jupiter".
Sigh. Read more carefully...
(1) It says, quite deliberately, "MUST NOT refuse to accept a
message on that basis". That is _very_ different from the
excerpt you quote.
Sorry for being sloppy in implying that section 4.1.4 requires that receivers
accept "HELO this is Jupiter". The connection between cause and effect is
less direct. I should have said that this requirement (no rejection when
verification fails) makes the HELO name unreliable for authentication. It
means that receivers MUST accept mail from transmitters that provide a
fraudulent ID, i.e. one that does not correspond to the IP address of the TCP
connection. The fact that receivers can't rely on the HELO name leads to
little use of that name for authentication, and attempts to find some other
name (MAIL FROM) that will more reliably ID the transmitter. This may be the
reason so many transmitters provide useless HELO names like Jupiter.
The Jupiter example came from a long ago discussion in the SPF group, in which
the conclusion was we have to accept this, even if it is obviously invalid.
The reason was that some legitimate senders have HELO names like this, and we
would be rejecting legitimate mail.
Why not say:
An SMTP server MAY reject a mail session if a domain name argument in
the EHLO command does not actually correspond to the IP address of the
client. Clients that are unable to establish this correspondence MUST
NOT provide an invalid domain name argument.
How the correspondence is established should be left to another spec (CSV, SPF,
whatever). There is no change in the syntax of the HELO command, no change
that would affect receivers following the old MUST NOT rule, just a change in
emphasis that should have a beneficial effect on the reliability of HELO names
as IDs.
I see very little downside. The examples of the digital camera and the roaming
laptop don't justify degrading the reliability of HELO IDs for everyone else.
Maybe I'm not understanding these examples. See below for discussion of the
roaming laptop.
(2) The place where the names get "used", i.e., the SMTP client,
is covered by the strongest type of statement that we can make
in a standard:
"The SMTP client MUST, if possible, ensure that the
domain parameter to the EHLO command is a primary host
name as specified for this command in Section 2.3.5."
Then the strength of this statement is nullified with a MUST NOT in the next
paragraph.
The key case that justifies "if possible" is mentioned in the
next sentence, but there are others. 2.3.5 is very explicit,
arguably to the point of tedium -- the string must be
fully-qualified (no local names or incomplete strings permitted)
and it must be resolvable (by any plausible reading "in the
public DNS" -- and I wrote myself a note in December to ask
where things had gotten weird enough that we had to add that
phrase).
(3) "HELO this is Jupiter" fails the syntax rule
helo = "HELO" SP Domain CRLF
in several ways. If you receive such a string, you are
perfectly justified, conformant, and in line with the spec to
reject it with a 501 code. Indeed, you would be outside the
spec if you did not do so. You wouldn't even have to figure out
whether "this" was a resolvable FQDN or not.
The prohibition against an unauthorized domain name should be just as strong as
the prohibition against bad syntax. The fix is equally simple in both cases,
even for digital cameras with built in transmitters.
My imagination doesn't stretch nearly far enough to get from
those three points to "seems to encourage using HELO/EHLO names
that are useless for authentication".
That prohibition in 4.1.4 goes back to RFC 1123, which predates
2821 by nearly a dozen years. There are a number of reasons for
it. Some of them are subtle, some of them involve cases that we
would now prefer to see handled via submission servers and prior
agreements. But let me give you a contemporary example that
might make the situation clear. Suppose there is a laptop with
a full service, conforming, SMTP client on it. It sends mail,
manages queues, etc. -- no submission server involved. It has a
permanent name, say roaming.example.com. However, it is not
connected at a fixed point to the public Internet. It roams,
and may have different addresses on different networks on
different days. If its owner is being really careful she even
uses dynamic DNS to be sure that, at any time the host is
online, its domain names points to its actual address. Or, if
she knows that the host is used at only a half-dozen locations
and has a usual address at each, she might set up static DNS
entries that list each of them as its address. Now suppose a
message is sent to you from that host, after which it is shut
down. When it reappears on the network a few hours later, it is
at a different address. If you resolve the domain name at that
time, you get a valid address for the host, but it isn't the
same address the host had when it initiated sending the message.
Do you really want to reject the message? If you do, please
don't justify the decision on the basis that that you have done
a careful and adequate job of authentication -- _nothing_ you
can do with HELO/EHLO as they are defined in 5321 is going to
get you to a point at which that is possible.
Let us call this the "roaming laptop" problem. There are several solutions.
1) The laptop user can leave her setup as is, relaying through the same server
as when she is in her home town.
2) She can publish a DNS record authorizing a collection of IP addresses, or
even entire blocks that she might be using.
3) She can use an address literal, saying in effect - please accept this
session request without a HELO check.
For the typical "road warrior" running Windows and Outlook, {1) is by far the
most likely solution. Even if she changes the default settings for SMTP
server, she will still need to keep the same POP or IMAP server, and the same
password to access her home mailbox. Same for the other popular email
programs. They are just not expecting to use a local SMTP server.
Solution 2 is not yet available, but if senders were to start taking HELO names
seriously, a solution would quickly develop. It would be something like SPF,
without the risky or ambiguous terms that have led to lack of universal
acceptance.
Solution 3 is no worse than the status quo. There is no degradation in service
for those using this method, if others chose a more reliable method.
The purpose of changes should be to avoid lost mail, as
opposed to rejected mail, which doesn't have any damaging
effects on reliability. SMTP REJECT should be encouraged as
the proper response to doubtful requests. (Sorry, we don't
accept mail from unidentified transmitters.)
2821/5321 were carefully written to give you the option of
making that decision... or the decision to not accept mail from
even-numbered IP addresses for that matter. I personally cannot
recommend it unless you have a much better notion of
"identified" than anything you can get out of HELO/EHLO. I'd
also suggest to you that the typical user who is waiting for a
message to arrive is not likely to distinguish between
"rejected" and "lost". Unless the former causes the sender to
get on the phone, the only issue is "didn't arrive". Your users
and experience may differ.
The difference between rejected and lost is critical to the reliability of our
email system.
So what are you asking for that isn't there already?
Elimination of the IP address literal option? Some criteria
for "verifiable" that would presumably lie well outside the
scope of SMTP?
If you can deprecate HELO, surely you can do the same for
address literals.
Independent of some practical issues and the fact that address
literals can sometimes actually provide useful information, that
would accomplish what?
Eliminate what seems to be useless information, and motivate senders to provide
useful information instead. What is in an address literal that we can't get
more reliably from the TCP address?
You can already make a policy decision
to reject any message that starts with EHLO and an address
literal; doing so would be consistent with your position that
false rejects are ok as long as one doesn't drop the message.
I can't recommend that we try to dig into the question in the
standard, but consider whether you would prefer:
EHLO [#.#.#.#]
where "#.#.#.#" is a valid, public, IP address that might even
be static and stable and have a legitimate reverse mapping to a
host name, and
EHLO comment.example.org
where "comment.example.org" is a valid, registered, resolvable
FQDN but the only information at its DNS node is
IN TXT This is a mobile host and I can't really tell you
what or where it is without compromising my privacy.
You don't have much "authentication" information either way; you
might be able to extract as much or more contact information
from the first.
I would split the mail into two streams. 99% of legitimate messages are sent
from transmitters with verifiable HELO IDs. 1% have an address literal, or
some more explicit way of saying "I can't ID just yet, please accept this mail
session anyway."
Idea: The HELO name could actually specify what authentication method the
sender offers in lieu of a robust HELO check.
EHLO _dkim.hostname.example.com
Do keep in mind that we know from experience that the bad guys
have absolutely no trouble obtaining perfectly valid domain
names.
I agree that reputation requirements should be outside the
scope of 5321. Same for the details of any particular
authentication method. What I am suggesting is not that level
of detail.
I know you can't turn back the clock and make radical changes
to SMTP, but in revising the spec, we should do everything we
can to make up for the original deficiency in the HELO command
and the lack of attention to security issues.
I think that, had you made this point at the time (others did,
more or less), the DRUMS WG would have told you that the changes
from the language of 821 about that field to the requirement for
a resolvable FQDN _and_ the addition of the IP address literal
option for SMTP clients that didn't reliably know their own
names, was "everything we can to make up for...". It seems to
me that you are looking for something else entirely, either to
push the Domain name that is expected to appear in that field
beyond its plausible limits (unless you are willing to start
prohibiting some legitimate cases) or to replace the Domain name
with an authentication token or identifier. And that is why I
suggested in an earlier note that, if you or others really think
that would be useful, you start writing an extension spec for an
authenticator that would supply the information you want, rather
than trying to overload it onto EHLO. Of course, you'd probably
then have to adopt a policy of rejecting (or assigning a very
low score to) any message that came from a source that didn't
adopt and support your extension, but that is how it goes with
twenty-odd years of installed base.
Writing a new draft now would be a waste of time. A discussion like this would
be the first step, and I am not seeing in this group any desire to change the
status quo, or even any recognition that there is a problem. That is the major
hurdle with all attempts to fix the identity fraud problem on the Internet.
The status quo is profitable for many, and at least acceptable to most others.
It's not a technical problem, but a problem of motivation and organization.
The technology is there if we really want it.
I appreciate your time also, and I will try to keep this discussion focused on
the HELO ID.
-- Dave