Re: [ietf-smtp] IETF Policy on dogfood consumption or avoidance - SMTP v

Peter,

Thanks for your comments and wisdom.

I agree, we have to judge on how a strict or relax a system is.   I 
follow the Postel's principle and did so before it existed.
You mentioned the space issue in MAIL FROM.  Yes, an old biggie with 
some MUAs, like OE.  Some servers enforced the syntax, others, like 
wcSMTP, does not.
There are others items, like how the a SMTP server handles DATA that 
doesn't include the minimum  RFC 822 or 2822/5322 headers where the 
minimum requirements differ.  There are, of course, applications where 
headers do not exist or not those related to email, i.e. print, fax, etc.
The issue with minimum TIMEOUT values, scaling and load management and 
the whole idea that the server MUST wait until the client issues a 
QUIT, never close the socket, especially after a transaction is 
completed and the MTA waits for more mail to send. wcSMTP gives you 
far less than 5 mins to start a new transaction, otherwise <click>.
So there are certainly many areas of mail system that we fine tune, 
But I think in principle no one is hurt by the fine tuning.  If it 
did, rest assured, I will nix the idea even if it just hurt 1 system. 
 1 system too many.
But in the regards with the specific HELO/EHLO [IP-literal], I firmly 
believe the flat out usage rejection in horribly wrong for one simple 
reason - you MUST change your otherwise valid code to comply.
I don't see any justification for the SDO to flat out reject it.

I can see justification if the SDO checked for a client IP match, like 
wcSMTP does, but it did not.
I can see justification if the SDO did a PTR lookup, found records and 
rejected for that reason, but it did not.  Thus reason would be more 
acceptable and would probably pressure me to adjust because the specs 
does use MUST use FQDN when it is available.
So Peter, I don't know if we are not agreeing, in my opinion, this 
specific issue is much different that any other.  It actually forces 
changes in MTA code to make sure it no longer uses ip literals.  IMO, 
that promotes changing the specs.
Finally, I am not a regex guru, my wcSMTP C/C++ CheckHello() function 
does a complete syntax check:
//-----------------------------------------------------------------
// 451.2 05/14/04 01:51 pm
// return TRUE if helo is a domain name or if a dot address is
// equal to the connection address

BOOL isGarbage(const char *sz)
{
    if (sz) {
       for (DWORD i = 0; i < strlen(sz); i++) {
         if (sz[i] < 32 || sz[i] > 'z') return TRUE;
       }
    }
    return FALSE;
}

BOOL IsNumber(const char *psz)
{
    if (!psz || !psz[0]) return FALSE; // zero length string
    char *pOut;
    strtol(psz,&pOut,10);
    return psz!=pOut;
}

//------------------------------------------------------
// smtpFilterCheckHello
//
// Check the HELO/EHLO (szHelo) hostname for IP literals, [#.#.#.#], 
against
// the connecting client ip (cip) address.
//
// return FALSE if syntax or match fails, TRUE otherwise.

BOOL APIENTRY smtpFilterCheckHello(const char *szHelo, const DWORD cip)
{
    DWORD addr = INADDR_NONE;
    if (szHelo && szHelo[0]) {

       //----------------------------------------------
       // check for space
       const char *p = strchr(szHelo,' ');
       if (p) return FALSE;
       if (isGarbage(szHelo)) return FALSE;
       //----------------------------------------------

       // 454.2F9 check for illegal [] formats
       if ((szHelo[0] == '[') && (szHelo[strlen(szHelo)-1] != ']')) {
           // illegal format, left, but no right bracket
           return FALSE;
       }
       if ((szHelo[0] != '[') && (szHelo[strlen(szHelo)-1] == ']')) {
           // illegal format, right, but no left bracket
           return FALSE;
       }

       if (szHelo[0] == '[') {
           char sz[MAX_PATH] = {0};
           strncpy(sz,&szHelo[1],sizeof(sz)-1);
           char *p = strchr(sz,']');
           if (!p) {
             // illegal format, no right bracket
             return FALSE;
           }
           *p = 0;
           addr = inet_addr(sz);
           if (addr == INADDR_NONE) {
               // illegal dotted-decimal address
               return FALSE;
           }
       } else {
          const char *p = strrchr(szHelo,'.');
          if (p && IsNumber(p+1)) {
              // illegal TLD
              return FALSE;
          }

          addr = inet_addr(szHelo);
          if (addr != INADDR_NONE) {
              // illegal format, no brackets
              return FALSE;
          }
       }
    }
    return (addr == INADDR_NONE) || (addr == cip);
}

Yes, I agree this check is deterministic because it would follow the 
specification.  The handling part is probably were the disagreement 
might be.   I choose to use a spec-related rule. The SDO choose to use 
some non-SMTP subjective rule to reject ip-literal usage entirely. 
That is not SMTP, imo.
--
HLS


On 12/20/2019 1:55 PM, Peter J. Holzer wrote:
> On 2019-12-20 12:07:58 -0500, Hector Santos wrote:
> > On 12/20/2019 9:11 AM, Peter J. Holzer wrote:
> > > I am very unclear on what you understand to be "deterministic".
> >  From google:  Deterministic functions always return the same result any time
> > they are called with a specific set of input values and given the same state
> > of the database.
> >
> > Nondeterministic is basically when the function need more help beyond the
> > basic protocol.
> >
> >  From google: In computer science, a nondeterministic algorithm is an
> > algorithm that, even for the same input, can exhibit different behaviors on
> > different runs, as opposed to a deterministic algorithm. There are several
> > ways an algorithm may behave differently from run to run.
> >
> > Nondeterministic systems contain fuzziness in results. The key is that, your
> > results MAY be different from mine.  Not so with deterministic functions,
> > protocols and methods.
> So how is
>
>      if re.match(r'^\[\d+\.\d+\.\d+\.\d+\]$', ehlo_string):
>          return REJECT
>
> non-deterministic? This depends on only a single input parameter which
> is completely under the control of the client. Send the same EHLO
> command and you get the same response - always.
>
> OTOH, you suggested "checking the remote IP address of the connection
> against the address literal in the EHLO command" and "checking whether a
> PTR record exists" as deterministic rules.
>
> These not only have more input parameters (the remote ip address and the
> result of a DNS query respectively), those are subject to temporary
> changes: The IP address may change because of routing changes and NAT
> gateways, DNS responses may be lost or outdated. So while the algorithm
> itself may be deterministic, the result is not, because some input
> parameters are prone to random fluctuations.
>
>
> > > You seem to be making assumptions on what a "good operator" should do,
> > > use statistics to verify that applying these assumptions as rules have a
> > > sufficiently high true positive and false negative rate and then call
> > > them "deterministic".
> > Wow, really, is that how you read my text?   It conflicts with my
> > philosophy.
> >
> > Sorry, no. In fact, deterministic concepts really do not need the operator
> > to be involved to make a decision. He just enabled it or not.
> Well, enabling a specific check or not is a decision. Nobody expects a
> human to be directly in the loop during an SMTP transaction.
>
> > It is the protocol negotiation violation that is 100% repeatable,
> > everywhere, that makes it popular and workable.
> There is no protocol violation. There is your (the developer's) personal
> opinion on how a sending MTA on the public internet should be
> configured. I may personally agree with them, and you might even do a
> survey and find that "97 out of 100 experts agree", but it's still a
> personal opinion, not part of SMTP as specified in RFC 5321.
>
> You apparently keep stats and check them periodically, which is good and
> - as Keith so forcefully pointed out - is more than many ESPs do. I
> don't say that what you are doing is wrong - in fact it seems like a
> quite sensible thing to do - but it certainly doesn't match your claims.
>
>
> > > If other people do the same (but with assumptions that don't match
> > > yours) you call it "subjective".
> > Yes, when you have different yields, yes, its nondeterministic, and I would
> > say subjective and also presents a security problem.
> You seem to have missed my point. My point is that whether you call
> something "deterministic" or "subjective" seems to depend on whether you
> came up with the idea or someone else instead of any actual determinism
> of the rule. (See example above).
>
> > With this SDO being the only site I am aware of that flat out rejects
> > ip-literal usage regardless of validity, right not, it is subjective rule on
> > that part only.  You will not find that to be that case with 99.99% if the
> > sites.
> Just as subjective as your rule that the peer address must match the IP
> literal, or your suggestion that an IP literal must onlybe used if there
> is no PTR record. Both may be good practice, but they are not mandated
> by the RFC.
>
> And even for outright syntax errors you might choose to tolerate them.
> For example, the RFC is quite clear that there is no space between ":"
> and "<" in "MAIL FROM:<foo(_at_)example(_dot_)com>". Yet enforcing that rejects
> quite a lot of legitimate mail (or did, when I last tried it). So there
> is an assessment of the pros and cons of being conservative or liberal
> involved.
>
>          hp
>
>
>
> _______________________________________________
> ietf-smtp mailing list
> ietf-smtp(_at_)ietf(_dot_)org
> https://www.ietf.org/mailman/listinfo/ietf-smtp

_______________________________________________
ietf-smtp mailing list
ietf-smtp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf-smtp