ietf-smtp
[Top] [All Lists]

Re: [ietf-smtp] [dmarc-ietf] RFC 7372 on Email Authentication Status Codes

2014-09-21 12:58:47

On 19 Sep 2014, at 00:44, Ned Freed <ned(_dot_)freed(_at_)mrochek(_dot_)com> 
wrote:


On 17 Sep 2014, at 22:41, <ned+dmarc(_at_)mrochek(_dot_)com> 
<ned+dmarc(_at_)mrochek(_dot_)com> wrote:

Hi,

On 09/16/14 05:08, Murray S. Kucherawy wrote:
       URL: https://www.rfc-editor.org/rfc/rfc7372.txt

This document registers code points to allow status codes to be
returned to an email client to indicate that a message is being
rejected or deferred specifically because of email authentication
failures.

This document updates RFC 7208, since some of the code points
registered replace the ones recommended for use in that document.

I hope this is not off topic here:

(chair hat on)

It is. I've cc'ed and directed replies to the ietf-smtp list, where this 
sort of discussion belongs.

(chair hat off)

RFC 7372 proposes to use a 550 response code for reverse DNS auth
failures, see section 3.3.

Correct.

Reverse DNS checks are usually done early in the connection (like IP
blocks) in the connection establishment stage of the SMTP dialog.

RFC 5321 allows only a 554 error response there, see section 4.3.2.

You're misreading RFC 5321. Nowhere does it say that its lists of possible
responses are exhaustive. So it is perfectly permissible for RFC 7372 to
specify additional possible responses as long as they fit into the overal
theory of reply codes.

It doesn’t say that (well-known) reply codes can be chosen arbitrarily in
whatever SMTP stage as long as they fit some formal schema either.

The publication of an RFC specifying a new code or a new usage of an old code
is hardly arbitrary.

What theory of reply codes?  THE theory of reply codes goes back to FTP
conventions and the 70s which were adapted to SMTP and HTTP and
other transfer protocols.   5yz is the well-known code for a permanent 
negative
completion, and x5z, the category code, that meant “protocol defined” in FTP
includes “policy defined” in SMTP.

Exactly.

So suggesting code 550 with its semantics is perfectly okay for any SMTP
stage but CONNECT because RFC 5321 and its ancestors fortunately are pretty
clear when discussing reply codes and sequencing.

But that isn’t the point.  The question is whether SMTP clients SHOULD also
expect a “policy code” in CONNECT and MAY act accordingly, and how this
is covered by said RFC.  My concern aren’t MUAs who might wonder what
colour to select for some doesn’t-work-popup but border MTAs where
semantics and interoperability matters.

If the server returned a 550 error on CONNECT, the client has three options:

(1) Immediately bounce the message, which is what it should do.
(2) In violation of protocol and common sense, treat this as a temporary
  error and retry until the message times out.
(3) Something else even more bogus.

So where's the problem? It seems the worst case scenario is that the client
retries unnecessarily and continues to fail to get through until the message
times out. And even this seems very unlikely.

Is that all you consider when defining a new reply code or redefining an old
code for a new purpose?  If a client in its protocol engine suddenly starts
misbehaving then someone has to hire a better software engineer.  Please,
let’s not discuss trivia.  Mail systems are more than having a mail bounced,
deferred, or seeing it delivered happily, since 20 years and more.

Why do you say that reply codes in the first, second, and third digit actually
matter and at the same time that clients won’t look at the last two?  Of
course they do, all over.  No mail system beyond a certain size can do
without looking at reply codes directly or indirectly.  Reply codes can be fed
into reputations systems to execute anti-spam, anti-abuse measures in
automatic, semi-automatic or manual ways.  Reply codes are used to
monitor the reputation of outbound IPs.  What happens if the grey IPs
turn black because someone has missed to register a new reply code?
He can watch his customer care department blow up: all legitimate mails
are bounced.  And what if white IPs turn black?  He needs a good
lawyer because bounces mustn't be bounced and so he suffers serious
mail loss.  I’d call that an interoperability issue.  Shop systems are
trying to figure out what exactly happened to their billing mails, bulk
mailers are using them, etc.  Heck, people even have been parsing
the reply messages themselves, because neither base code nor
extended code gave them enough information.  You’ve no idea, what
those people have been costing us in customer care and postmaster
support.  There are weird and not so weird application of reply codes.
Personally, I’d advice anyone against using reply codes directly, and to
always only use what’s absolutely necessary, but sometimes it’s all people
have got, and their use is legitimate, they are part of the interface!

Interfaces are to join two sides neatly at the protocol level, and in
between is the machinery that makes the system tick.  That machinery
is simple and negligible if you only always think of the cute little
sendmail installations of the '80s with their 1000 mailboxes; it is
complicated and vast in the large mail systems like ours that serve many
millions of domains: information flows both ways from the interfaces into the
machinery and back from there controlling the interfaces.  Information
goes from MTAs and MSAs into the system out to the border MDAs
and vice versa.  It’s not that _we_ depend on reply codes within the
machinery but others might!   SMTP, the protocol, just is the (inter-)face
of it all, and reply codes are its eyes.  So whenever you do something
to the base RFC that _is_ the interface specification, directly or indirectly,
you have to tell people about it.  And the only way you can do this is
to put that information in the RFC itself, RFC 7372 in this case, rather
than making it a “(mailing list ) eyes only” affair that only popped up here
by chance.



Different from the reply message, the reply
code is not just informal because in the real world, semantics come with
system behaviour.  We are fighting the battle of fending off bad reputations
while maintaining ours every day.

Well, if we're going to talk about the real world, do you have any evidence
of clients that misbehave when they receive a 550 response in a banner
as opposed to some other 5yz response? Indeed, do you have evidence of
any client that looks at anything other than the first digit in a banner
response?

I'll also note that if we had wanted to stick to previously specified usage, 
we
would have been stuck with 554 or 521, both of which carry the semantic of "no
SMTP service here", which to the extent that second and third digit choices
actually matter, seems far more likely to be problematic.

As for facts rather than evidence, see above.

As for code 550 specifically, no, I don’t have evidence.  No one _is_ using
it in the banner.  People have adopted 554 base code with or without
5.7.1 extended code many years ago, and so have we in the ‘90s without
issues ever since.

As for code 554: Am I the only one who doesn’t have that much an issue with it?
It doesn’t just say “No Service”, being a “policy code” it says, “No service 
for you
unless you come back under different conditions on your the sender’s side."  
That’s
how those codes are read.  Now what can those conditions be, in the banner?
The IP itself or anything associated with the IP, a bad DNS PTR RR, an RBL
entry, etc.  I could also live with any other code "55z Go Away You Stinks” in 
the
greeting were it properly specified.   On the other hand I wouldn’t want see
“45z You Stinks But I Might Change My Mind”;  that seems not to be useful
as the reputation of an IP does't change by retrying with the same IP.  A few
unfortunately very large mail service providers are using 421 in banner for 
these
rejects.  It is non-sense and backfires, filling our queues by the terabyte and
so 45z would have an interoperability issue also.

Back to RFC 7372.

Should we choose not to implement code 550 in CONNECT, would we be
violating RFC 7372, and by reversal conclusion, per your argument also
RFC 5321?  I don’t think so.

(1) Other than changing the base RFC itself there have been only two options
so far to get it updated: (a) specifying an SMTP extension, or (b) registering
new extended reply codes with the IANA database.  RFC 7372 obviously isn’t
an SMTP extension that needs a new EHLO keyword; it does the latter and
nothing else.  That’s fine.  So no change to the base RFC beyond the  extended
reply codes.

(2) Nowhere it says that 550 should also apply to the banner stage directly or
indirectly.  Or do you mean form actually here also is function?  Do I have to
read the section heading as the precondition that leads to the extended code
and then to the “associated” base code?  So “associated” actually means
“mandatory”?  That’s interesting.

(3) Nowhere it says that 550 in the banner should have the same special
sequencing rule like 554 the only other policy code specified by RFC 5321.
If that doesn’t matter, why have it a MUST in RFC 5321?

By not making an interface specification clear you won’t get it implemented
the way it was intended.  You are risking interoperability issues and you are
weakening earlier RFCs and the overall system.  People again will be
inventing their own codes or use existing codes in whatever stage they see
fit _beyond_ RFCs with the exact single argument that is the golden rule of
the theory of reply codes.  That won’t do.  We’ve been there.  No, thanks.

I have talked a lot about reply codes;  I should have talked more about
interfaces and their meaning because the real issue isn’t even the reply code
itself, interoperability or not, but the way you handle that interface change!

I appreciate the efforts of RFC 7372.  It is a good RFC but it cannot be
understood per your intention.  We are implementing those RFCs,
we are operating the systems, we are the backbone to the net, we have
to know about interface changes that _might_ affect the critical parts of the
system so it keeps on ticking 24/7. 

Christoph

_______________________________________________
ietf-smtp mailing list
ietf-smtp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf-smtp