Re: SMTP QoS

Howdy,

Having thoroughly overdosed on fireworks, and trying to focus on the 
substantial issues, and being surprised how many there are...

I.      LACK OF SEMANTICS

At 03:07 PM 7/1/2001, ned(_dot_)freed(_at_)mrochek(_dot_)com wrote:
> Now, none of this implies that priority cannot be carried in a header.
        Perhaps the most basic issue is that the specification
        is standardized syntax, without any standardized
        semantics.

        The specification says that each MTA can do whatever it
        feels like with the message.

How can we possibly expect any sort of Internet-wide useful service out of 
this?
-----

II.     LIKELIHOOD OF ABUSE

> > The past experience you have been citing pertains to LOWER priority.  That
> > is quite different from one which includes HIGHER priority.
> Only one instance of past experience pertains to LOWER priority. Other
> instances pertain to higher priority, and there has been no indication of
> substantial abuse.
I apologize for misunderstanding the examples you were citing.  Worse, I am 
astonished to hear that Internet mail has been functioning with a mechanism 
-- even a non-standard one -- that would permit me to specify increased 
handling priority.
The fact of that ignorance prompts me to suggest that the reason we do not 
have a track record of abuse for this feature is because the general 
population of users does not know about it.  I am, of course, stating that 
explanation vastly more tentatively than I believe it.

> >   As I noted, we have well
> > seen that the Internet is now under the strong influence of selfish
> > actors. They do not participate as good neighbors.
> One more time: Past experience directly contradicts this (IMO overly
> simplistic) assessment.
Alas, I wish that this were simplistic or, at least, just a matter of my 
own opinion.  And since it has been 25 years since I made a habit or 
reading the relevant research literature, I can't counter your disbelief 
with a mass of citation.  Somehow I suspect that assuring you the citations 
are there won't be convincing either...
Just for fun, I did a single sample query of a Cognitive Psych PhD and they 
could not quite believe that anyone would assert that this sort of 
mechanism (zero cost to the requester) would not be abused.
The fact is, however, that the Internet today sees a considerable amount of 
abusive behavior.  Some is intentional and some is through 
ignorance.  Either way, a mechanism that is open, cost-free, and might have 
substantial benefit for the user WILL get used.
And abused.

It will get used excessively by some percentage of the population.  Will it 
get used enough to undermine the global mechanism?  (And remember that this 
particular mechanism has no solid specification of semantics, so the 
freedom of choice by each relay is particularly large.)
I claim yes.

And I claim that we have more than enough history with email abuse to 
substantiate that belief.

> anything that stands out, like a flood of mail from a single source all 
> marked priority urgent, is a sign of spam and is likely to be used as a 
> means of filtering mail.
The flood is the warning.  The priority is not.

In fact we all frequently see postal direct marketing folks emulate 
priority packaging, as a scam to get you to look at the contents.  Why do 
you believe that won't happen with email?

> Priority labelling used as a honeypot, in other words. But unlike you, I 
> doubt they are that stupid.
Sorry, but spammers are but one example of folks who will abuse the 
mechanism.
An open, cost-free mechanism that provides better service provides an 
inherent incentive for over-use.  The psychology of this point is not even 
slightly controversial because the incentives are far too clear.  And 
over-use will destroy the benefit.
-----

III.    TRANSPORT PRIORITY AS SMTP OPTION

> However, this discussion does lead me to conclude that priority is best 
> done as
> an SMTP extension.
Given that the goal is transport handling I entirely agree.

(In case it has been looking as if I am against having any mechanism, let 
me try to be clear that I think having one is essential.  The rest of the 
communications world has priority mechanisms, so we need them in email 
too.  Business use demands it.  My concern is that we specify something 
that will work.)
-----

IV.     DEPLOYMENT OF INTER-OPERATOR SMTP AUTH

> > > > 2.  References to usage of SMTP Auth with this option are confusing,
> > > >     since it is used for posting, not relaying.  I'd appreciate
> > >   SMTP AUTH not only provides facility for posting, it also
> > > provides facilities for relaying authentication credentials.
> > Do you really expect that relaying across the
> > open Internet will require use of SMTP Auth?  In which century?
> Nobody said anything about requiring it for relaying across the open Internet.
> Using it as means of facilitating the use of extensions like priority
> is another matter.
This is getting confusing.  You don't like my use of the word "require" yet 
you accept that use of Auth for "facilitating" a priority extension.
If the priority extension is to be seriously useful -- and for Internet 
mail to satisfy business requirements, it does need to be -- you claim that 
the Auth mechanism will enable, facilitate or otherwise be relevant to 
creating that utility, then I claim it will need to be Internet-wide.
(Can you only send FedEx to places with which you have made a prior 
arrangement? Bi-lateral agreements are a good way to start, but they don't 
scale, of course.)
My point, therefore, is that wide-scale, inter-organization use of security 
has been essentially non-existent for anything except SSL, and then only 
for privacy, not authentication.  That makes it highly problematic to 
specify a facility that relies on inter-group authentication.  (The current 
proposal does not do that; but your argument seems to rely on it.)

> I'm going to turn one of your own arguments back on you here. You like making
> analogies to lower layers.
Ned, this is worth being clear about:  my use of lower layers is not for 
"analogies".  I like looking to the history of lower layers engineering 
because it so often points up issues and approaches to solutions for the 
transport aspects of email.  Both are packet switching.  Vast differences, 
yes, but core similarities.
Staying on the high protocol ground, EDI provides an interesting tidbit of 
relevant experience:  Here we have a service that is commercial 
messaging.  In fact, it does have QoS.
But only within single providers.

No QoS guarantees between providers.  And that was with customers willing 
to pay remarkable premiums for good service.

> Well, one of the things that happens all the time is
> private bilateral agreements between service providers to provide improved QoS
In fact, bilateral agreements are the essence of the public international 
phone service and have always been the essence of the public, commercial 
Internet.  No ISP is ever forced to peer with another.  And peering is a 
bilateral action.
But your point is, of course, about TAILORING the service on a bilateral 
basis.  And it's just fine that a) it happens, and b) the terms are kept 
private.
However, note that we still do not have a way to specify QoS at the lower 
layers and that all efforts to provide that capability over the last 30 
years have failed.
With luck, the experience with private arrangements is providing useful 
input to the IPv6 QoS work.  Otherwise, the v6 effort will suffer the same 
likelihood of being a theoretical, inadequately specified service that does 
not match real needs.
Just like this proposal for email.


> At present there's no reason to do anything similar based on SMTP AUTH 
> because there aren't any facilities tied to its use. But if there were, 
> who's to say it would not happen?
Again I am thoroughly confused.  This hypothetical, future, inter-provider 
use of SMTP Auth is entirely independent of the proposal under 
discussion.  If not, then where does it appear in the specification?

> > If my request for special handling (priority) carries no expectation of
> > what actual handling will take place, then the option is entirely
> > useless.  If it does correlate with actual handling, then I had better have
> > better knowledge about the delivery behavior of my message than I had
> > without the option.  That constitutes increased predictability.  That is
> > what quality of service work is about.
> And actual experience disproves this line of reasoning completely. People will
> use such facilities even without even a scintilla of evidence of their
> efficacy.
Huh?  So the spammers will use it whether it works or not.

But you believe that regular users will regularly and differentially use a 
feature, even  if they have no idea whether it does any good?  Is this 
supposed to be a cargo cult explanation of system use?  They saw an email 
once get delivered and it had a priority bit set.  So they keep setting the 
bit hoping that, someday, their own email will get delivered quicker?

> This is why I find the arguments made in the FAX WG relating to the
> unreliable nature of MDNs to be unpersuasive
And I'll gladly swap bar wg anecdotes about the variable and creative ways 
people analyze protocol behavior and use, in the Fax wg and 
elsewhere.  However I'm unclear how to apply your point to the current 
discussion.
Please remember that my highest-order concern is in fact the first major 
section in this note:  no standardized semantics.
-----

V.      INTRANET

> > > > 4.      The IETF does not do intranet standards.
> > > Actually, the IETF does intranet standards all the time. This is 
> > especially
> No, what we conclude is that the notion of the IETF not doing intranet
> standards is complete hooey now and always has been.
Though probably frustrating, this exchange has actually been pretty useful.

It is clear that the oft-uttered claim "the IETF does not do protocols for 
intranets" is entirely the wrong wording.  And it is clear that small 
attempts at more precise wording, such as my earlier comment:
> > It is about being limited to an operational infrastructure that has 
> > significant dependence on a single, coherent administrative control.
Is equally inadequate.  The reference to 'infrastructure' was trying to 
distinguish between end-point behavior vs. intermediaries.
I am now suspecting that the whole point hinges on the IETF protocols 
having a) universal semantics, and b) data transfer performance 
characteristics that must work over the Internet as well as intranets.
Media-specific tailorings, such as ARP and DHCP using a broadcast 
mechanism, clearly do not work over the Big I.  However neither do they 
have different semantics from one intranet to another.
Bilateral agreement on semantics is an EDI speciality.  It's not the soup 
the IETF stirs.  Yet is is, in reality, the stock of the current proposal.
d/

----------
Dave Crocker  <mailto:dcrocker(_at_)brandenburg(_dot_)com>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464