ietf-smtp
[Top] [All Lists]

Re: Has the IETF dropped the ball?

2005-03-09 13:06:23

Russ,

Thanks for your thoughtful comments.

At 02:50 PM 3/8/2005 -0800, Russ Allbery wrote:

David MacQuigg <dmq(_at_)gain(_dot_)com> writes:

> The public is getting mad as hell about spam.  There *will* be a
> solution to this problem.  If the IETF doesn't provide it, some
> politicians or bureaucrats will.  I may not understand all that is going
> on in the IETF, but it sure looks like they dropped the ball.

Your confusion may be cleared up by realizing that the IETF, like most
standards organizations, does far better at clearly documenting the right
way to do something after that's been hashed out over time than it does at
making up a new way of solving a brand new problem.  The latter is more
properly termed "research and development" and is something that works
most effectively in small teams of smart people rather than in large
committees worried about getting all the details right.

I agree that IETF shouldn't be doing the R&D. Their role should be more as impartial judges of the proposals that have been laid before them. Where I think they failed in this case of email authentication methods is in letting the advocates take over, and not insisting that at least one of the proposals be the "common elements" that are needed by all of the methods. That would be almost nothing for an end-to-end method like DomainKeys, and a few common items for SPF and SenderID that share a need to communicate through a chain of forwarders. All the other details could left to the competing groups to do as they wish.

By leaving everything to the competing groups, we have a situation where a forwarder using SenderID will add the SUBMITTER parameter, and the Destination using SPF will not know what to do with it. Surely there can't be endless debate over how to communicate this piece of information.

Given that no clear technical solution has yet emerged from the pack on
dealing with spam, it doesn't surprise me at all that the IETF has not
been able to pick a clear winner and say "everyone should do this."

Notice I didn't say IETF should pick a winner right now between SPF, SenderID, and DomainKeys. I certainly can't pick one. What we need is a very simple framework that will allow the industry to move forward and not worry that they will have to change everything when a winner does emerge. A company with a popular spam filter, for example, should know by now what authentication headers will look like. They don't need to know *how* the authentication was done, just that there will be a common format for the results.

> They have allowed the standards effort to disintegrate into a bunch of
> programmers squabbling over details that are not needed in a universal
> standard.

I don't think all those people are just stupid and squabbling for no
reason.  I don't think we've figured out a great solution and are just
failing to implement it because we're fighting over trivia.  Rather, I
think people have proposed a huge number of solutions that range in
quality from hideous to barely mediocre, without a *good* solution in the
lot, and much of the arguing that you're seeing is over whether it's
worthwhile to try to implement one of the several barely mediocre
solutions and deal with the resulting breakage because nothing better
seems to be appearing, or to wait for more R&D.

I don't think anyone in these discussions is stupid. I do think the problem is that many are so caught up in the heat of battle, that they can't even think about compromise on the few shared items that everyone needs. There may also be a few who feel that such a compromise would be to their disadvantage, and would prefer that the deadlock continue. The IETF should be above all this, and work only for the public good. Listen to the advocates. Understand their objections. But don't give them a vote.

I don't see anything in the shared items I have in mind that would "break" anything. These are simple items like how to communicate to the next station the domain name that was authenticated. We could add a few words to existing Received: headers, but to avoid any risk of breaking something, the standard could call for a new Authenticated: header. The standard could specify such things as when to add an Authenticated: header (when crossing into a new administrative domain). This would be the same for all methods.

> If we continue the present path, it may be years before there is a clear
> "victory" in the battle between competing and incompatible standards.

And how do you think that the IETF should pick among the current bunch?
Drawing lots?  The reason why there's no victory is that none of them are
clearly better, and they all take very different approaches.

See above.

The fundamental problem of spam is that we want to use a system
specifically designed to not use authentication to either reject all mail
from "bad people" (where the definition of who's bad varies wildly from
one person to another) or to instead reject all "bad mail" (with a similar
wild variation in definition).  This isn't exactly an easy problem.  This
is certainly not a problem that we understand well enough that all that's
needed now is to get a few experts in a room and write the final draft.
For one thing, it's not even clear that it's *possible* to retrofit
authentication into the e-mail system in such a fashion that doesn't break
so much of e-mail that it would be better to just use a different protocol
entirely, and that's just one of the many issues that arises.

Mixing in the question "What is spam?" is one way to bring all progress to a halt. Luckily we don't need to answer that question. We just need to provide a common protocol to handle the results of a Recipient having decided something is spam. Part of the challenge for the judges here is to recognize what is relevant, and limit debate on issues that aren't.

Yes, people are frustrated, but there is little correlation between how
frustrated people are and how easy the problem is to solve.

My frustration is partly due to the lack of progress on what I see as simple problems. I'm also well aware that what I see as simple, may in fact have some hidden difficulties. I listen very carefully to discussions on this and other lists, looking for those little bits of knowledge that will clarify something I didn't understand. I can't just automatically believe it when an expert says I am wrong, however. I much prefer to investigate and reach my own conclusions.

There are parts of the authentication problem that are very difficult, but I'm not seeing those difficulties in the part that needs to be a shared standard. Establishing a chain-of-trust, for example, will be essential for all IP-authentication methods, and could turn out to be an insurmountable problem. That problem need not be solved by the IETF, however. If you focus on just the problem of how to communicate authentication results from each forwarder to the final destination, you will maximize the chance that one or another group will make their IP-authentication method work. With no standard there will be endless finger pointing over the reason for failure of transfers involving forwarders with competing protocols. With a clear standard, it will be possible for one contender to demonstrate success even if a competitor's system is a reluctant participant.

-- Dave