(keith, this message is also meant to serve as an answer to your private mail)
Unfortunately, casting the issue in terms of saving or discarding SMTP
is also unhelpful. It makes sure that we debate lots of protocol
details, without having any basic agreement on the more import usage
and protection details that will drive the mundane technical choices.
well, but, so, the debate about protocol details was raging before i
started saying "stop trying to save smtp" so these items are at best
unrelated.
On the other hand, a relevant point about current operations, versus
future operations, is how we move 100 million users. But again, that is
best discussed in terms of the users and their usage, rather than the
protocol details.
So: What is this better service supposed to look like?
see further below, which was written in response to private mail from
keith before i saw dave's message (to which this is a direct reply.)
How do we transition a very large installed base to it.
the whole installed base is in incredible pain right now and there will
be a lot of sites like mine who will switch over early, turning off
their tcp/25 service, as soon as we think there are reasonable
alternatives for reaching us. (this contrasts with the ipv6 transition,
where the existing ipv4 installed base is not wholly in incredible pain;
only a few mega-networks are feeling incredible ipv4 pain, and the vast
majority of non-mega-networks is feeling no ipv4 pain at all.)
Extra credit: The transition question has more to do with user
motivation and choice than with technical engineering.
how well i know that! like i said, the motivation is overwhelming. an
alternative only needs to barely work before it will be seen as compelling.
that means i'll be cutting off tcp/25 while most of the world has only that
way, or fax and phones, to reach me. thousands, no, tens of thousands of
other sites will do likewise. the mega-exporters of batched interpersonal
communications traffic will be the last to change over, since their costs
are the highest, but they (hotmail, aol, etc) will be forced to endure these
costs when the "reachability base" dwindles.
in other words, by waiting until the barn is on fire and the horses are
dying, the quality and stability of our alternative, and the costs of getting
moved to it, have become almost irrelevant. i guess we can all pat ourselves
on the back now, for letting things get this bad, before we acted. (NOT.)
i am damned close to turning off tcp/25 even if the only alternative is
telco and fax and netnews and web. THAT is how the pain:gain ratio is now.
SO, telling me that moving 100M users is hard, won't make me believe it.
--------
now as to the question keith and dave both asked: "why does smtp have to die?"
i guess it could be done on top of esmtp but we'll have to be able to
put sig(heloname,ipsrc,privatekey) in the HELO, sig(fromaddr,privatekey)
in the MAIL, and crypt(rcptaddr,publickey) in the RCPT. the receiving
relay should be able to determine, at esmtp session time, whether the
owner of the heloname'd server knows what ip address it is using and
that it knows it is sending this mail and that there is transitive
trust from it to you. similar restrictions on the MAIL fromaddr.
we'd want to sign the recipient name using its public key to indicate
that the sender has a way of learning/reaching this key. and so on.
but at smtp session time, the smtp speaker must be able to determine a
shared value system between sender and recipient, which means the
recipient's values must be known to the smtp speaker. things such as
"i only accept e-mail that is promised to be solicited or non-bulk"
have to be able to be matched up with promises of the form "i promise
that this mail is solicited" or "i promise that this mail is not bulk".
the combination of authentic promises and transitive trust is enough
to ensure that all communications is consensual, which is an explicit
goal now that the internet community no longer has this as an implicit
assumption (due to its originally small size and closed nature.)
so, whether we use esmtp as a bearer channel is indeed irrelevant, so
long as we'd be willing to stop trying to maintain a distinction between
an MTA and MUA, and perform recipient-specified functions at session time,
and so long as we can add attributes to all the existing protocol elements.
but those are HUGE elements of the (e)smtp model, and the resulting protocol
can in my opinion be called "new" even if it looks superficially like what
we do now. there's absolutely no reason to use the same port number, for
example. there's no fallback. if newproto doesn't work because the firewall
or nat or whatever only allows tcp/25 sessions, then give up and use the
telephone or fax machine.
transitive trust and mailbox/server/relay certificates and chains of same
are the much harder issues, technically. pgp and x.509 are both HARD for
the majority of mailbox owners (think of your parents of grandparents) to
use. to make a new global-scale system, we have to make it marketable.
--------
now as to my own motives:
my reason for harping on the "stop trying to save smtp" issue in recent
days here is just my disbelief that baynesian or other filters, or lists
of who's been naughty and who's been nice, or anything like that, can be
bolted on "usefully". that stuff is all crutches and bandaids while what's
needed is a whole new monster.
my reason for making "all communications must be consensual" as the key
design criteria for a new interpersonal batch communication system is that
it's what i've always believed, from the earliest days of the first RBL which
later became MAPS, and it's the single thing that's most failed in the (e)smtp
model as we've exploded the endpoint population to the hundreds of millions.