ietf-smtp
[Top] [All Lists]

Re: [ietf-smtp] Reducing minimum recipient limit?

2019-12-16 15:01:31
On 12/16/19 3:29 PM, Brandon Long wrote:


On Mon, Dec 16, 2019 at 5:45 AM Keith Moore <moore(_at_)network-heretics(_dot_)com <mailto:moore(_at_)network-heretics(_dot_)com>> wrote:

    On 12/16/19 8:01 AM, Tony Finch wrote:

    this an area where practice appears to have diverged from the standard.
    I not keen to see the limit reduced, but perhaps advice to clients to
    default further below 100 than they may doing already may be warranted.
    I can only speak to Postfix, which defaults to sending up to 50 and
    accepting 1000.
    I would like a requirement that servers must not reject the entire message
    if there are up to 100 RCPTs even if the server returns 452 for some of
    them. Senders can then pipeline safely.

    Part of me wonders why there's a minimum number of RCPTs at all
    if, practically speaking, the client has to be able to deal with a
    server-imposed limit no matter where it is.

    Seen in this light, the minimum requirement seems like more of a
    guide for optimization by clients than a requirement for servers.

    Then again, if existing clients are currently assuming 100 or some
    substantial fraction thereof, I would not want to break them by
    telling servers that they can now impose whatever limit they like.

    I am tempted to suggest that there be a MAXRCPTS EHLO response to
    let the server advertise its explicit limit.  But again, this
    would give license to servers to change existing limits.

    Other than the rare server that really cannot store 100
    recipients, is there some operational reason to permit fewer?  
    Did someone empirically determine, or arbitrarily decide, that
    spambots would be more likely to send 100 recipients than
    legitimate senders?

The limit that Gmail imposes is not based on a specific number of recipients.  Instead, it's based on available resources and the in-memory footprint of the rules necessary for handling each recipient.  GSuite allows for creating rules that can be made complicated by admins who do things like try to implement their own anti-spam system, or write tools to convert from their previous system to ours in less than ideal conversions, as well as having different rules per recipient... and then claim there is no way to reduce their rules, regardless of how un-useful they are (see the other thread about ip-literals in EHLO except writ large). Couple that with some less than ideal implementation for the rules system itself (probably fine for small sets of rules, but very painful for large sets) and you end up with some worst case scenarios in the 50MB per recipient range.

There's an open bug to remove that particular resource based temp failures as we've evolved the system to reduce the problem, including properly 4xx'ing recipients that have different rules (before there were complicated attempts to merge the results or accept messages and generate bounce messages later if there were mixed results), so we may go back to accepting the rfc standard minimums most of the time next year.

That said, multiple addresses per transaction has been a pretty rare feature.  I'm sure our workload is biased because of how rarely we send multiple transaction mail ourselves (see also enterprise rules engine on sending with different rules per recipient), but last I checked the average per transaction was 1.1.  As we've had to move more computation to be at smtp time, some of which is per-recipient, including too many recipients in a transaction increases the number of operations that have to complete successfully, some of which may be cross oceanic (you think these two addresses are in the same domain, but they actually work for different parts of an international enterprise which has different data geoloc requirements for the recipients in different offices), increasing the chances of a failure which will temp fail the entire transaction. Keeping the number of operations small and contained leads to more predictable performance and better fault isolation.

Ie, its cheap to say "send this to all these people", but much harder to know whether every one of those people can accept it.

Brandon


Reading this, it appears to me that this is basically a tradeoff between reporting per-recipient failures during the SMTP session versus reporting (some of) them in via one or more NDNs.   Of course NDNs have various costs, not only to generate and transmit them, but also in accepting a lower probability of the errors actually being reported to some party that can address the issue that caused the delivery failure.

Regarding what the standards prescribe, my sense is that the standards should give implementors a lot of leeway, but should be constructed as to /maximize the reliability of successful delivery of (legitimate) email/.    (Granted that "legitimate" is hard to define but I think it has to do with two broad factors - one is whether the content of the message is appropriate, and the other is whether the message is correctly representing its sender and recipients and signal path; and NOT anything to do with things unrelated to message content, say, how many recipients were in the envelope or whether an IP address literal was used in EHLO.)

So does a hard requirement on how many recipients a server should be able to accept, improve or degrade the reliability of email delivery?   I'm not sure, but I immediately see two consequences of not having a minimum requirement:

1.  having a minimum recipient requirement appears to shift some of the error reporting to NDNs, with lower reliability for such reporting.

2.  not having a minimum requirement may shift some implementation burden (and potential for errors) from servers to clients (by requiring clients to deal more flexibly with SMTP error codes), perhaps resulting in more errors occurring earlier in the signal path with consequently less probability of messages being delivered.   Alternatively, the client can behave like qmail and only relay one recipient per envelope.   This makes the client code simpler and less error-prone, but it also consumes more network resources, client resources, and server resources, which might also have an adverse effect on reliability if resources are limited at any point on the signal path between client and server.

I suspect that all of these considerations pale in comparison with the adverse effect on reliability caused by bogus spam filters (which seems to be part of the reason for not supporting a minimum number of recipients per envelope).   So I might be inclined to suggest that anything that encourages more message filtering via dubious criteria should be discouraged by future revisions of the standards and/or by any operational recommendations we might someday publish.

Keith


_______________________________________________
ietf-smtp mailing list
ietf-smtp(_at_)ietf(_dot_)org
https://www.ietf.org/mailman/listinfo/ietf-smtp