ietf-smtp
[Top] [All Lists]

Re: Last Call: SMTP Service Extension for Content Negotiation to Proposed Standard

2002-07-01 22:53:11

[referencing draft-ietf-fax-esmtp-conneg-02.txt]

executive summary:  I recommend  against approval of this document
withut extensive changes.  It's not well thought-out (except for the
rather narrow application of SMTP to a one-recipient fax gateway)
and it seems likely to degrade SMTP interoperabliity if widely
adopted.   I do suggest some changes which would make it less
harmful, but it still needs considerable work to be of general 
applicability.




1. layering problems

this functionality is a poor fit for SMTP because SMTP is the 
mail *transport* protocol, and as such should not be dealing 
with content at all, which is a user agent function.

I admit that (by design) there is no opportunity for e2e interaction
at the user agent level in the current protocols, and also that any 
mechanism that might be used to advertise user agent capabliities 
(e.g. rescap) introduces the opportunity for that mechanism to get 
out-of-sync with the actual capabilities/preferences of the recipient.
but what's to stop this mechanism from getting out-of-sync also?
it seems like this extension has very limited applicability - it will 
only work reliably when the SMTP server and user agent or gateway are
very closely tied and when the recipient always uses the same UA
or UAs of very similar capabilities.  (certainly not true of me!)

one can argue that extending SMTP to allow it to perform negotiation on 
behalf of the user agent is no worse than putting that negotiation in a 
separate protocol.  but if we're going to extend SMTP to perform functions 
on behalf of the user agent, it needs to be made very explicit that this 
negotiation is being done on behalf of the user agent - and in particular
that the negotiation being requested represents the preferences of the 
*user*  (i.e. the sender and/or recipient) rather than some third party.

IIRC, this is the same thing that was insisted on for the OPES stuff - 
the negotiation/transformation needs to have explicit consent from, and 
be under control of, at least one of the ultimate endpoints
(sender or receiver) and it seems to me that it's appropriate to apply
the same policy for email.

2. protocol design problem

for transformations that should be under control of specific recipients,
the protocol is poorly designed.  in particular there seems to be no
ability to apply content-transformation on a per-recipient basis,
to handle the case where different transformations are acceptable to
differnet recipients.  again, this drastically limits the applicability
of the protocol (it won't work well unless all recipients at that
SMTP server have the same capability set, which is very unlikely for
most conventional SMTP servers)

of course it's not easy to get SMTP to handle different client-side
transformations for different recipients, since SMTP is designed to 
transfer the exact same message body (modulo received fields) to each 
of a set of recipients.  this is a different model than used by
phone-based facsimile protocols which expect there to be only one
recipient of any given transmission, so it's not surprising that
the fax-style negotiation doesn't fit well into SMTP.

in order to make client-side negotiation a clean fit into SMTP, it is 
necessary for the client to first query the server for the capabilities 
of each intended recipient, then to deduce how many total transformations 
are necessary, then to send one or more transformed versions of the message.
for instance, if recipients A, B, E, and F could accept one transformation 
of the message and recipients B, C, and D could could accept a higher 
quality transformation of the message, the client might attempt to send the 
high-quality version to B, C, and D and then send the lower quality 
version to A, E, and F.

but as currently designed the proposed extension doesn't have the 
necessary protocol hooks to permit this - at least not without
a considerable amount of ugliness.  overloading the RCPT 
command to return capabilities breaks the RCPT command and forces
the client to do RSET in order to send different versions to 
different recipients.

3. timing

this proposal could not have come at a worse time.

putting content negotiation in the transport protocol is likely 
to result in a significant increase in anomolous behaviors
from the mail system which are difficult to diagnose and fix -
and the proposal is being introduced at a time when the mail system 
is already suffering from serious reliabliity problems (often due 
to ill-conceived measures for blocking spam).  at this time,  
*any* proposal for adding new functionality to SMTP should be viewed 
with strong reservations, unless it somehow offers an improvement
in reliability.  a proposal that significantly increases the complexity
of SMTP error handling seems unlikely to do that.

4. naming and applicability 

the protocol claims to be a general-purpose content-negotiation scheme
but actually has "fax" assumptions embedded very deeply. 

first of all, it assumes that there is a linear ordering relationship
for document suitability to the recipient.  while this might be arguable 
for facsimile images (even this is debatable given orthogonal attributes 
like resolution and color), it's certainly not the case for MIME
email in general.

5. recommendations

a.  I strongly recommend that you redesign this extension so that, instead
of overloading the RCPT command, you define a separate command to
request a recipient's capabilities.  e.g.

RCAP <recipient-addr>

would return the conneg capability associated with that recipient,
if available.  there would need to be separate indications
for permanent error (perhaps if the recipient addr were invalid),
temporary error (e.g. can't do the database lookup),
no capabliities available (perhaps not an error, just no information),
and a return of capabilities.

one nice side-benefit is that it doesn't change the error handling
model, because the proposal mentioned above doesn't affect 
the actual message handling commands at all.  (however it does
add a few round trip delays)

b. the idea that the sender should send the "best" or "highest
quality" version needs to be relaxed or generalized, since there's
currently no clear notion of what this means. (and thus no way to 
know whether an implementation conforms to these criteria).  

c. this extension needs to be made less fax-specific.
it needs to be examined with some realistic non-fax-specific
test cases, and it needs non-fax conneg examples also.

d. there needs to be a good way for the recipient to specify
capabilities to the SMTP server, at least for any SMTP server
that can serve arbitrary UA recipients.  probably this means another
extension along with SMTP AUTH.

still, this is a big can of worms, because there's never been
any ability of a UA to specify something to its inbound SMTP
servers before - there's presently not even a good way for
a UA to know which server(s?) should be given this information...

e. make it absolutely clear that such transformations are only to be
made with explicit authorization by the sender and/or recipient.

(a dedicated smtp-to-printer hardware device would be considered 
to have explicit authorization simply because it was purchased
for such a purpose, but a SMTP server acting on behalf of a user
using a general-purpose MUA would need such authorization) 

f. the document needs to consider the effect of message forwarding
on the returned string - if a recipient has his mail forwarded
elsewhere it's not clear what conneg string should be returned.

g. the document alludes to the ability to work through intermediaries
but this isn't explained anywhere, and it needs to be explained.


6. specific problems

4.2 Client Action:
          
     If the server issued a 250-CONNEG, as part of its
     EHLO response for the current session, the client MAY
     issue the CONNEG parameter with RCPT-TO.
          
     If the client issues the CONNEG parameter with
     RCPT-TO, then it MUST honor the capabilities specified
     in the CONNEG RCPT-TO reply, and transform data that is
     sent, so that the target can accept the data. The
     client SHOULD transform the data to the "highest" level
     of capability of the target.

this incorrectly assumes that (a) there is only one target when
the client can easily specify multiple RCPT commands, and
(b) there is a clear notion of "highest level of capability"
for the target.

also why is the term "target" used rather than the more traditional
"recipient" or "mailbox"?

     If the server rejects the RCPT-TO command with a 404 reply, the 
     client may later reissue the RCPT-TO with the CONNEG parameter.

presumably in the context of a different SMTP session?
it's not clear...

4.3 Server Action:
          
     If the client specifies "CONNEG = REQUIRED" in the RCPT-TO, 
     but the target does not support the CONNEG parameter, the target
     MUST reject the RCPT-TO command with a 504 reply. 

here there is a potential conflict between various extensions.
a RCPT command may fail for any number of reasons.  if any of those
reasons is a permanent fatal error it should be sufficient for 
the server to report any one of those errors.  it's not reasonable
for this extension to pre-empt other error notificaitons.
(it may be that other extensions have similar wording, in which case
they're similarly flawed)


     If the target support the CONNEG parameter, but it can not return 
     the recepient's capability temporarily, the target MUST reject the 
     RCPT-TO command with a 404 reply. For example, if the target get 
     the capability information from a directory, but its connection is 
     offline, the target MUST reject the RCPT-TO command with a 404 
     reply. 

similarly: if any permanent failure is detected in the RCPT command, 
the appropriate 5xx response SHOULD probably be returned in preference
to any 4xx response.  for instance if the recipient's mailbox has
been administratively disabled then a 5xx response should be returned
rather than a 404 response which would mean "ask me later what kind
of media types  this recipient can accept"... whereupon the client
would do so only to (eventually) find out that it couldn't deliver 
the message to the recipient anyway.

     If the client specifies "CONNEG = OPTIONAL" in the RCPT-TO, the 
     target MUST process the address and message as if the requested 
     CONNEG capabilities had not been specified.

this doesn't seem to make sense as written.  is "CONNEG = OPTIONAL"
really supposed to be interpreted the same as omission of the 
CONNEG parameter?

also, I don't think there should be spaces around the = sign -
I don't think this is legal syntax for SMTP extensions.

     Regardless of the value of the parameter, if the target does 
     support the CONNEG parameter, then it MUST issue a 250 reply, 

again, there could be other reasons for permanent or temporary failure
of the RCPT command, and the assertion of a CONNEG parameter should 
not pre-empt the reporting of such failures.

     followed by the capabilities of the target that is specified by 
     the RCPT-TO address. Successful responses to CONNEG RCPT-TO 
     requests will always be multiple SMTP lines.  The first line 
     is the normal RCPT-TO response, and subsequent lines beginning 
     with the exact string "250-CONNEG " and "250 CONNEG " are 
     the CONNEG responses. The last line begins with "250 
     CONNEG".  

I wonder here - assuming that this can of worms does not get closed
back up, and there might later be demand for other mechanisms
to request recipient capabilities (e.g. since this one is not very general),
might there be multiple kinds of strings emitted in the RCPT response?
e.g. CONNEG and CONNEG2 and FINAL_SOLUTION_CONNEG?
and if so should this document insist that CONNEG be last?

     If the SMTP server supports ENHANCEDSTATUSCODES, the response 
     strings for a success are "250-2.1.5 CONNEG" and "250 2.1.5 
     CONNEG". The response strings for indicating a permanent 
     failure are "504-5.3.3 CONNEG" and "504 5.3.3 CONNEG". 
     The response strings for a temporary failure are "404-4.3.3 
     CONNEG" and "404 5.3.3 CONNEG".

are these codes already defined in 1893?  (I didn't look) 

I don't think it's appropriate to dictate extended status codes
since their entire purpose is to provide more precision than
the SMTP reply codes.  there are various sources of permanent
failure and various sources of temporary failure and the set
of status codes is extensible independently of SMTP reply codes.

     All CONNEG-capable clients and CONNEG-capable servers MUST
     be able to successfully process CONNEG lines that are up to 512
     characters long, as required by RFC2821. If the length of CONNEG
     lines is greater than 512 characters, the server MUST insert
     line breaks and make next CONNEG line. 

CONNEG *what* lines?  command lines? response lines?  both?

     The contents of the capability listing MUST conform to
     [RFC2506], [RFC2533] and [RFC2534], and MAY be used to emulate
     services, such as the facsimile start-of-session capabilities
     negotiation as described in "Content Feature Schema for
     Internet Fax". [RFC2879]




5.   SYNTAX
     
     Command with "CONNEG":
      "RCPT TO:" ("<Postmaster@" domain ">" / "<Postmaster>"
                   / Forward-Path) (SP "CONNEG =" ("REQUIRED" /
                 "OPTIOANL") CRLF

the space doesn't belong after CONNEG, and OPTIONAL is misspelled. 



     Reply:
      ( ("250-" CRLF) *("250-CONNEG" capability CRLF)
        ("250 CONNEG" capability CRLF) )/
      ( ("250-2.1.5" CRLF) 
          *("250-2.1.5 CONNEG" capability CRLF)
           ("250 2.1.5 CONNEG" capability CRLF) )/
      ("504" CRLF) /
      ("504 5.3.3" CRLF) /
      ("404" CRLF) /
      ("404 4.3.3" CRLF) /

it's not appropriate to put literal response codes in the grammar,
particularly the extended status codes which might actually need to
vary more than is indicated here.

much better to have something like:

reply: initial-reply *(conneg-reply/other-reply) final-reply

initial-reply: reply-code "-" [ status-code ] CRLF

conneg-reply: reply-code "-" [ status-code " " ] "CONNEG" capability CRLF

other-reply: reply-code "-" [ status-code ] *char CRLF

final-reply: reply-code " " [ status-code ] CRLF 


      capability = <<as per [RFC2506], [RFC2533], [RFC2534]
                    and [RFC2879] >>



6.   EXAMPLE

     An example of ESMTP sequence with successful RCPT-TO response

     S: 220 ifax1.jp IFAX
     
     C: EHLO ifax1.jp
     
     S: 250-ifax1.jp
     S: 250-DSN
     S: 250 CONNEG
     
     C: MAIL FROM:<May(_at_)ifax2(_dot_)jp>
     
     S: 250 <May(_at_)ifax2(_dot_)jp> sender ok
     
     C: RCPT TO:<June(_at_)ifax1(_dot_)jp> CONNEG = REQUIRED
     
     S: 250-<June(_at_)ifax1(_dot_)jp> recipient ok
     S: 250 CONNEG (&(image-file-structure=TIFF-minimal)
     S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
     S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
     S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
     S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
     S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
     S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
     S: (ua-media=stationery) )

this needs clarification - it looks like the sender is sending
multiple lines without a 250 or CONNEG on each line.
it would probably be better to break up this example into
several SMTP lines and also to explain that any of those lines 
can be longer (up to XXX bytes) than shown in the example.

     
     C: DATA
     
     S: 354 okay, send data
     
     C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
          Per:
          (  image-file-structure=TIFF-minimal
             dpi=400
             image-coding=JBIG
             size-x=2150
           )
        >>

this is confusing - it makes it appear that TIFF-FX
is the only content-type acceptable.  

also TIFF-FX is not the name of the content-type,
presumably it's application/tiff-fx or some such.

     
     S: 250 message accepted
     
     C: QUIT
     
     S: 221 goodbye


     An example of successful RCPT-TO response when the length of 
     capability is greater than 512 characters. 

     S: 250-2.1.5<June(_at_)ifax1(_dot_)jp> recipient ok
     S: 250-2.1.5 CONNEG (&(image-file-structure=TIFF-minimal) ...
     S: 250-2.1.5 CONNEG   .....
     S: 250 2.1.5 CONNEG   (color=Binary)


     An example of succssful RCPT-TO response when CONNEG-capable 
     server supports ENHANCEDSTATUSCODES.

     S: 250-2.1.5<June(_at_)ifax1(_dot_)jp> recipient ok
     S: 250 2.1.5 CONNEG (&(image-file-structure=TIFF-minimal)
     S: (MRC-mode=0)(color=Binary)(|(&(dpi=204)
     S: (dpi-xyratio=[204/98,204/196]) )(&(dpi=200)
     S: (dpi-xyratio=[200/100,1]) )(&(dpi=400)
     S: (dpi-xyratio=1) ) )(|(image-coding=[MH,MR,MMR])
     S: (&(image-coding=JBIG)(image-coding-constraint=JBIG-T85)
     S: (JBIG-stripe-size=128) ) )(paper-size=[letter,A4,B4])
     S: (ua-media=stationery) )

again, this response is confusing because it's line wrapped



     An example of ESMTP sequence with parmanent failure RCPT-TO 
     response. 

     S: 220 ifax1.jp IFAX
     
     C: EHLO ifax1.jp
     
     S: 250-ifax1.jp
     S: 250-DSN
     
     C: MAIL FROM:<May(_at_)ifax2(_dot_)jp>
     
     S: 250 <May(_at_)ifax2(_dot_)jp> sender ok
     
     C: RCPT TO:<June(_at_)ifax1(_dot_)jp> CONNEG = REQUIRED
     
     S: 504 <June(_at_)ifax1(_dot_)jp> recipient ok

     C: QUIT
     
     S: 221 goodbye



     An example of an ESMTP sequence with temporary failure RCPT-TO 
     response when the value of parameter is "REQUIRED":

     S: 220 ifax1.jp IFAX
     
     C: EHLO ifax1.jp
     
     S: 250-ifax1.jp
     S: 250-DSN
     S: 250 CONNEG
     
     C: MAIL FROM:<May(_at_)ifax2(_dot_)jp>
     
     S: 250 <May(_at_)ifax2(_dot_)jp> sender ok
     
     C: RCPT TO:<June(_at_)ifax1(_dot_)jp> CONNEG = REQUIRED
     
     S: 404 <June(_at_)ifax1(_dot_)jp> recipient ok

     C: QUIT
     
     S: 221 goodbye
     .
     .
     .
     retry according to implementation



     An example of an ESMTP sequence with temporary failure RCPT-TO 
     response when the value of parameter is "OPTIONAL":

     S: 220 ifax1.jp IFAX
     
     C: EHLO ifax1.jp
     
     S: 250-ifax1.jp
     S: 250-DSN
     S: 250 CONNEG
     
     C: MAIL FROM:<May(_at_)ifax2(_dot_)jp>
     
     S: 250 <May(_at_)ifax2(_dot_)jp> sender ok
     
     C: RCPT TO:<June(_at_)ifax1(_dot_)jp> CONNEG = OPTIONAL
     
     S: 250 <June(_at_)ifax1(_dot_)jp> recipient ok
     
     C: DATA
     
     S: 354 okay, send data
     
     C: <<RFC 2822 message with MIME Content-Type:TIFF-FX
          Per "A Simple Mode of Facsimile Using Internet Mail"
          RFC2305
        >>
     
     S: 250 message accepted
     
     C: QUIT
     
     S: 221 goodbye



7.   IANA CONSIDERATIONS
     
     On publicatiom of this document by the RFC Editor, the IANA 
     shall register the Content_Negotiation ESMTP extension defined 
     in section 3.


8.   SECURITY CONSIDERATIONS
     
     This ESMTP option calls for a respondent to disclose
     its capabilities.  Mechanisms for determining the
     requestor's authenticated identity are outside the
     scope of this specification.  It is intended that this
     mechanism permit disclosure of public information;
     hence there is no particular need for security
     measures.
     
     However there is nothing to prevent disclosure of
     sensitive information that should receive restricted
     distribution.  It is, therefore, the responsibility of
     the disclosing ESMTP server to determine whether
     additional security measures should be applied to the
     use of this ESMTP option.

seems like you should at least mention SMTP AUTH. 

     A man-in-the-middle attack might change the capabilities reported 
     for a given recipient. For example: Suppose the sender knows the 
     recipient has the ability to view color documents so they mark 
     some things in red in what is otherwise a black and white 
     document. But someone interferes with the returned capabilities, 
     indicating that the recipient only supports black and white. The 
     document is duly downgraded, with the result that the recipient 
     doesn't see what the sender marked. 

the STARTTLS command could be useful in providing integrity assurance.

     An indirect exposure can occur when the report of a capability 
     implies use of specific software.  If that software is known to 
     have security weaknesses, the capabilities report effectively 
     advertises the associated opportunity to exploit the security 
     weakness.
     
     For target SMTP servers that require security mechanisms to be in 
     force at the start of the session, the target SHOULD refrain from 
     including the CONNEG parameter in an EHLO response until the 
     requisite security mechanisms are in force.