On May 19, 2011, at 3:43 PM, Rolf E. Sonneveld wrote:
On 5/19/11 5:44 PM, Jeff Macdonald wrote:
On Wed, May 11, 2011 at 7:43 AM, Keith Moore
<moore(_at_)network-heretics(_dot_)com> wrote:
I'm not following this thread closely, but I thought I'd say something
about extended status codes. Part of the idea of extended status codes is
that you should be able to determine the likely source of the problem by
looking at the second facet of the status code. Or to put it another way,
the second facet of the status code is supposed to indicate _who_ probably
needs to fix the problem (e.g. the sender, the MSA, a relay, the delivery
agent, etc.)
From the SMTP transaction point of view there are only two parties involved:
the SMTP client and
the SMTP server
Yes, but we're not talking about a code that changes any action to be taken by
an SMTP agent here. The only thing that SMTP does with respect to enhanced
status codes is (if the server supports the appropriate SMTP extension) to
convey the enhanced status code from the server to the client. The SMTP
client looks only at the 3-digit reply code, not at the enhanced status code.
Furthermore many enhanced status codes aren't passed in SMTP messages at all,
but rather in DSNs. The real purpose of the enhanced status code is to report
to the sender of the message (envelope MAIL FROM or return-path) the specific
nature of any errors, in a language-independent fashion.
From the sender's point-of-view, there are many places where things can go
wrong, e.g:
- the sender may have caused the error, perhaps by mistyping a recipient
address,
- there might be an error in the formatting of the message that prevents some
intermediary or delivery agent from processing the message, (e.g. a mail
gateway or a spam filter that looks at MIME body-parts) (note that such an
error can be caused either by bugs in the sender's MUA or by bugs in some sort
of gratuitous translator downstream)
- there might be an error in the sender's MUA configuration (e.g. incorrect
domain name, username, or password for MSA)
- there might be an error in DNS or some other server used to govern routing of
the message
- there might be an error in mailer configuration in some relay between the
sender's MSA and the recipient's final delivery agent (e.g. when a MX record
points to an MTA that doesn't know what to do with mail for that domain)
- there might be a resource limitation in some relay (e.g. out of disk space),
a hardware failure (can't read spool file), or network failure preventing an
MTA from reaching any next hop
- the message might not meet criteria imposed by the recipient's mail domain
(e.g. message size)
- the message might not meet criteria specified by the actual recipient (e.g.
SIEVE filtering)
- the message might not meet criteria specified by some party with no direct
responsibility to the sender or recipient (e.g. reputation servers, blackhole
lists)
Each of these requires a different kind of fix, and probably, a different party
to fix it.
The enhanced status codes aren't perfect, and cannot be. It will always be
possible for some conditions to have multiple potential sources. e.g. if an
MX record points to an MTA that doesn't know what to do with mail for that
domain, is the problem in DNS or the MTA configuration? It could be either.
But if you're in a position of trying to determine why someone's mail isn't
working, and how to fix it, it helps if the error indications you're getting
back make as clear a distinction as possible between these different kinds of
errors. So for better or worse, the codes are organized in such a way as to
attempt to distinguish errors according to the likely source of the error.
So if the intention is to enable systems/people to determine who needs to
take action to fix something, we only need two 'subject' codes: it's either
the client that has to fix something, or the server.
See above.
The server can delegate part of its policy determination process to other
systems that provide input to it, but on the receiver side the SMTP server
remains primary responsible entity for the transaction. If the SMTP server
encounters problems in e.g. DNS blacklist lookups, or PTR lookups etc. it can
use the 'detail' part of the enhanced status code (3rd facet) to indicate
what the problem was (room for 1,000 different codes). Providing information
about a 3rd party in the 'subject' part of the ESC creates complexity: there
is virtually no end to the number of parties that can be involved in a single
transaction; apart from a 3rd party there can be a 4th party and possibly
even a 5th party and so on. To give two examples:
A user at company A (1st party) is sending mail to a user at company B (2nd
party). Company B is using a hosted AS/AV service, like MessageLabs (3rd
party) for sending and receiving mail. MessageLabs in turn, enables the
customer to use one or more public DNS blacklists (4th party) within it's
Dashboard function. Is it important for the MTA of Company A to 'know' about
the way the infrastructure of company B is designed and should Mlabs reflect
part of this in the ESC it returns?
Company A sends a newsletter mailing on behalf of Company B using a Company B
sender address. A receiving system at Company C, performs a Call Back
Verificiation (CBV) to the mail server of Company B. Company B in turn is
using a DNS Blacklist and finds that the IP address of Company B is listed.
What ESC code should Company C use in its communication with Company B during
the SMTP session?
I agree that the number of parties that can be involved in a filtering decision
is (sadly) unbounded. But I still think it's useful to distinguish between:
"This message was bounced due to configuration of a recipient's MTA" (in which
case you want to try to get the recipient's MSP to fix the problem), vs.
"This message was bounced due to criteria provided by a blacklist" (in which
case you want to try to get the blacklist to fix the problem).
In the latter case, going to the recipient's MSP isn't likely to fix the
problem any time soon. The MSP might reconsider using the blacklist if they
get enough complaints about it, but it will take a long time. Meanwhile,
they're probably just going to refer people to the blacklist.
Note that I never said I wanted to provide information about _which_ 3rd party
caused the message to be rejected in the 2nd facet / subject of the ESC. I
just think that there should be a single ESC subject code reserved for
rejections due to 3rd party criteria.
There is a myriad of possible topologies and interactions here. Let's keep it
simple: if (repeat: IF) the intent of ESCs is to make clear who should take
action, use two different subjects, X if the client needs to take action and
Y if the problem is on the server side. The 'Detail' in the ESC can then be
used to distinguish between different types of problems.
That's true only from a microscopic view involving a single client and a single
server. Reality is a lot more complex than that.
As Jeff already mentioned, history has learned us:
So it seems the intent and implementation have actually drifted since
Keith came up with the codes.
As RFC3463 is widely used it seems to me it's too late to use ESCs to
indicate which side of the SMTP connection needs to take action.
ESCs were never intended, and should not be used, to dictate any behavior for
SMTP agents. They were, and are, intended for senders.
And of course, some amount of drift is inevitable. But it still makes more
sense to try to preserve the structure of enhanced status codes than to just
discard it.
Why not simply use the x.7.y policy related codes and use the 'y' detail to
differentiate between the various situations? If we want, we can reserve a
range within 'y' for spam related reasons, like (2|4|5).7.[100-199]. But IMO
_documenting_ the various codes in an RFC is more important than the actual
codes we will use for this purpose.
Because doing so will conflate different sources of the problem. I don't want
to overload recipient-specified policy, recipient domain specified policy, and
blacklist-specified policy. Those need to be kept separate, because they need
different fixes. And it makes more sense to have a separate 'subject' for
each of these, than to try to distinguish the different sources of these
problems using different 'detail' codes.
Keith