This is the message to which Pete Resnick refers.
I did not get this until Pete mentioned the message number and I searched
for it.
It is the _last_ private message John sent to me.
In his other previous messages, John did not mention this experience.
--Dean
---------- Forwarded message ----------
Date: Thu, 29 May 2003 08:27:34 -0400
From: John C Klensin <john-ietf(_at_)jck(_dot_)com>
To: Dean Anderson <dean(_at_)av8(_dot_)com>
Cc: "presnick(_at_)qualcomm(_dot_)com" <presnick(_at_)qualcomm(_dot_)com>
Subject: Re: The utilitiy of IP is at stake here
--On Thursday, 29 May, 2003 00:05 -0400 Dean Anderson
<dean(_at_)av8(_dot_)com> wrote:
And I've been involved with SMTP, and its predecessors, for
over thirty years now -- standards, protocol design, gateway
design and implementation, and operations and deployment in
organizations and enterprises ranging from the very small to
the very large.
But have you ever tracked down an abuser?
Personally, yes, several times. And had partial responsibility
for that department for an ISP with millions of customers (both
dialup and dedicated) for a while. And trained the 20-odd
people in that department how to do it and what to look for when
the abusers were not on our network. And participated in
writing their terms and conditions.
Have you ever sold standards based services to customers that
want standards based services? Do you really understand why
customers want standards?
Yes, and I've written standards-based procurement specs too,
including for one of your former employers.
That's operational experience.
You seem to be unaware that Relays (open or closed) put the IP
address of the client in the message. This address cannot be
altered by the client. So it is impossible to send anonymous
email with an open relay.
I'm aware that they do it. I'm aware that, while doing it is
beyond the average spammer, IP addresses can be faked. I'm
aware that some spam software uses a two-hop method and that
only the Received address inserted by _my_ server --or a server
known to my server which I know I can trust -- is reliable. I'm
aware that spam often contains completely made-up Received
fields (although, obviously, not the locally-inserted one). And
I'm aware that getting from an IP address to a user
identification can be a very rough process, especially if the
user is coming from an Internet cafe on a cash-only, no-account
basis or using a public terminal in a library or the equivalent
that doesn't require or authenticate sign-ins, or using a dialup
service that makes no attempt to authenticate its users in order
to authorize their use (either, as in the Chinese case and a few
other developing areas, because they
make no attempt to authenticate or because they are operating on
an "internet calling card" basis, with cards that can be
purchased for cash and without identification.
I really have worked these problems, Dean. In the trenches.
The property of a users anonymity is not altered by SMTP.
In the sense that positive identification of a user who doesn't
want to be identified --even by species-- is typically hard with
any Internet protocol, that is correct. And I have _never_ said
otherwise. That is, however, where SMTP AUTH (and IMAP/POP
AUTH, and some uses of PKI, and maybe RADIUS, and proper use of
TLS/SSL with client certificates, and Kerberos, and...) come in
-- if I am running a server and I care who is using it (or some
resource on it) down to the user level, I need to require that
the users authenticate themselves and make sure they have the
necessary tools and credentials to do so to my satisfaction.
SMTP, and more especially its predecessors --but not SMTP AUTH--
was, incidentally, designed on what I think is exactly the model
you are assuming today. In that model, the identity of the
machine from which mail was sent could be authenticated (a
technical issue, hasn't changed much, although relays can make
things _much_ more difficult) and the operators of that machine
can be held responsible (administratively and legally) from
anything that originates from their machines. That
administrative and legal issue caused machine administrators to
either firmly explain AUPs, etc., to their users or to create
accounts and whatever they considered satisfactory authorization
mechanisms. The introduction of router-based LANs and desktop
machines with IP addresses and SMTP clients weakened that model
considerably, as did public internet "terminals" of several
different sorts.
SMTP abuse is not the only type of abuse that a user can send.
Of course not. On a given day, it is far more reasonable to
worry about various security-type attacks. But they are less
intrusive on the typical user and, typically, easier to defend
against.
If you reach different conclusions about something than I do
and we disagree, that is fine. But don't tell me what I
don't know or what I was thinking about, or assumed, at a
particular time.
Don't mean to tell you what you are thinking. What ever you
were thinking seems to be questionable, SMTP AUTH serves no
purpose, and solves no problem.
See "user authentication" above. You are entitled to believe,
and even to tell your customers, that such authentication is of
no use, but, for those of us who see it as an issue/problem, it
solves it.
The only problem SMTP AUTH *seems* to solve, judging by the
people promoting it, is that it is an alternative to
POP-BEFORE-SMTP. But of course, that was also a half-baked
idea, promoted by crusaders who thought (wrongly) that open
relays somehow (never explained) promoted spam, and somehow
permitted anonymous email, or enabled (non-existant)
multiplication. None of these harms ever withstood close
scrutiny.
Once again, regardless of what I think of Send-after-POP (and I
think it is an obscenity for a number of reasons), it is
legitimate for me to control or monitor (with real
authentication) who gets to use my servers. I may do that
because I want to be able to track down any user who actually
sends mail from my address range, or because I'm concerned about
allocation of scarce resources (do you know that some servers
are configured to impose different limits on the permitted size
of outgoing mail depending on which user is sending it?), or
because I'm compulsive (or delusional, depending on your
perspective) about property rights in the cycles on the server,
or because I've got some broken notion about attaching security
authorization levels to send different types of mail to the
client-server connection. The point is that each of the above
is a legitimate reason for wanting to authenticate an
originating user (not just a machine) to the first-hop SMTP
server. None of them require end-to-end authentication in a
multi-hop environment that doesn't exclusively involve trusted
servers. And you will note that I made exactly as many claims
that such authentication would solve spam as I did that it would
cure cancer... zero.
Wrt the spam issue, SMTP authentication is useful iff the
_originating_ server can be positively identified _and_ its
operators are cooperative but have no other reliable way of
tracing individual users. That combination rarely occurs in
today's environment, and I have never claimed (or believed)
otherwise. Never.
Real spammers don't abuse open relays.
Absolutely true, if you define "real spammer" as "one who
doesn't abuse open relays". I've got some evidence to the
contrary, using something close to your definition of a spammer:
back in my ISP days, we tracked down several spammers who were
violating our terms and conditions of service by sending the
stuff and, with the cooperation of law enforcement because of
the nature of the content being advertised, got to examine
software and "interview" them under oath. They were
deliberately using open relays (rather than, or in addition to,
the SMTP servers we provided) in an attempt to disguise the
origins of their mail, since they knew that sending it violated
the rules to which they had agreed. And their messages didn't
point to web sites or email addresses for responses -- they
pointed to phone numbers. Fortunately for us, they were also
stupid. E.g., if you buys telephone service from the same
entity from which you buy Internet service, it typically takes
that entity even less time to track you down by phone number
than it does to do so by IP address. The level of
sophistication has clearly gone up in the last several years.
The vast bulk, and by that I mean millions compared to a
couple hundred over a seven year period, of open relay abuse
was conducted by anti-spammer crusaders.
Look. I don't like the "anti-spammer crusaders" either, and I,
and server's I've been responsible for, have been on their hit
list more than once --either because they thought we were
running excessively open relays or because they thought we were
not being diligent enough in stopping/ prosecuting spammers.
I've also been targeted, in campaigns that are almost
indistinguishable, by spammers who were upset by efforts to push
back on them, statements of sympathy for those we caught and put
out of business, or because they didn't like our terms of
service and tended to enforce them. I've also had spammers
decide to "get" me by using my address in the From: and/or
envelope return path fields of some really nasty high-volume
bulk messages. These experiences are pretty unpleasant,
especially when they bring servers that support millions of
users to an approximation to a halt (and, yes, we learned to
deal with that, but it cost a lot). But I very much doubt your
statistics on an overall network basis, although I have no doubt
that they reflect your personal experience.
Our goal was to provide an relatively strong authentication
mechanism for a number of purposes, including
intra-enterprise address authentication (when used in
combination with strong internal administrative policies).
And what purpose does this serve?
See above.
I've worked for organizations large and small, ISPs large and
small, and have not seen any use for an authentication method
in SMTP.
See above.
An authentication method ties SMTP to the user. But SMTP is a
batch protocol. The user is long gone, in all but a few cases,
such as dialup. And _USERS_ aren't the only entities sending
mail.
As indicated above, it is mostly useful as a first-hop
authentication protocol. In first-hop arrangements the typical
uses of SMTP are not really "batch protocol" and authentication
of the user is meaningful. Now, some people have had the
notion that SMTP AUTH could be used to establish a chain of
trust among servers in a relay chain. I've never believed that
to be generally useful -- the authentication mechanism isn't
properly designed for passing along "the user claimed to be X,
and server Y authenticated that, and I trust Y because Z said
so, and I've got appropriate credentials Z and it was
authenticated to me" the information one would really want. And
it wouldn't scale well either, IMO. If one really wants
end-to-end authentication of messages or their senders, one
needs... well, end to end methods. And SMTP AUTH isn't an end
to end method and anyone claiming it is is delusional (or, in
your terms, "doesn't understand how SMTP works").
On the other hand, there is a lot of evidence that most
legitimate, non-bulk, Internet mail these days goes
local-client -> first-hop SMTP relay -> delivery SMTP
with the intervention of relays that are not under the
presumptive control of either the sender or the receiver. In
that case, the Received data recorded by the delivery MTA is
extremely useful, and user (or originator) authentication to the
first-hop relay may be quite helpful in identifying the actual
originator (or whomever has the credentials) whether in
conjunction with the address information it records or otherwise.
Yes, it helped with one small element of the spam problem, in
that it permitted authorized users of particular (not-open)
mail relays to authenticate themselves to those relays
How does this affect spam?
Dean, the irony of these exchanges is that you have been so busy
attacking me (and some others) that the possibility that we are
largely in agreement on the important issues has escaped you. I
am very skeptical, at best, about the possibility of a technical
remedy to spam that doesn't destroy Internet email in the
process. I think it is mostly a social problem, one that
becomes a legal problem if the social norms are violated. We
disagree about how to measure costs and impacts but, for better
or worse, the tide seems to be running in my favor. But, at the
point that it becomes a legal problem, even if only because
someone offers an "opt out" and then redistributes the
supposedly-removed address as a "validated address" to other
spammers, accurate sender identification is very important in
figuring out who to sue or prosecute.
But I, at least never expected that would do anything other
than to shift the spammers elsewhere and have essentially
said so on the list.
How does SMTP AUTH "shift spammers elsewhere"? Spammers are
the customers of ISPs. If those ISPs use SMTP AUTH, the
spammers will have that password. This changes nothing.
If an ISP sets things up so that it has _any_ reliable
user-identification mechanism (IP addresses work sometimes and
with some setups, and not others, and are problematic if one
wants the relay to be accessible to users coming from other
networks), and that includes per-user (rather than per-server)
credentials (names and "passwords", if you like), then it has a
positive way to identify who has originated a message that turns
out to be problematic. As outlined above, this is all about
sanctions for prior bad behavior rather than preventing the
behavior (except by the certainty of being caught and
inconvenienced). And the spammers go elsewhere, to places where
terms of service don't prohibit spamming, or where they are
clearly unenforced or unenforceable, because doing so is
rational. Why risk prosecution, high special fees, and service
disconnections (the least of the remedies in most cases) when
you can just find a more friendly home? Of course, if you
_can't_ find a more friendly home, that is where the open relays
and other tactics intended to hide the origins and path of your
mail set in.
regards,
john
--Dean