[Top] [All Lists]

Re: SMTP Transferred-By-Reference

2007-11-13 19:58:08

On Nov 13, 2007, at 10:03 AM, Dave Crocker wrote:
I'm feeling kinda dumb for having missed the ultimate flaw in the proposal.

The goal of the proposal is to permit deferred filtering analysis, or at least reputation analysis.

While being able to defer an assessment of reputation is a potential benefit, it is not the goal of this proposal. Millions of IP sources are identified as being untrustworthy when their ISP classifies these addresses as belonging to a residential access point. While these lists account for a sizeable amount of the spam being blocked (these systems are often compromised), an even greater amount of spam is blocked based upon dynamic reputations (often of their provider's servers). The game has changed. Handling a mixture of good and bad email must employ temporary 451 errors. : (

For ISP that host customers sending out a spam campaign, their servers may receive a temp error at that point in time. Unfortunately, this may cause some desired email to be delayed. Once spamming from that server ceases, normal acceptance is restored. This dynamic reputation tactic blocks often more than twice the amount of spam stopped by static lists. Graduated dynamic lists are available to suit the needs of different receivers.

Blocking an influx of spam using the temp error tactic frees up resources needed to process other (cleaner) sources of email. Simultaneously offering the TBR extension within the 451 4.7.8/9/10 temp error would provide the transmitter a means to immediately complete their transaction by reverting to the TBR mode. (Fewer transactions and less delay.)

The percentage of spam is increasing, and content filtering is being defeated on an ongoing basis. Reacting to new methods of obfuscation gives bad actors a significant resource advantage. The temp error tactic ensures this finite resource is not inundated to the point of collapse. The TBR mode would permit receivers a means to better prioritize their backlog of messages waiting to be scanned. The TBR mode would also provide transmitters a means to complete the SMTP portion of their exchange. The TBR extension would be a win/win for both the transmitter and receiver.

As someone else noted, most of this work requires the message header and/or content. That requires message transfer.

The TBR extension does not interfere with content checking. The TBR extension allows receivers to decide when they are ready to check content, and what has priority.

The information that is transferred during the retained SMTP exchange is minimally helpful, except for previous-hop IP Address. Everything else requires access to the actual message. This means reaching across the net to get the message for inspection.

Message content offers little assurance of its origination. Where to reach for the message is more significant. The TBR extension offers both an assured last hop IP address and a domain of origination. Often the origination of content is more important than the results of a scanning process. Receivers would be foolish to trust the results of content scanning alone.

And this is better than transferring the message during a regular SMTP session how?

A message transferred in the normal manner fails to validate a point of origination. DKIM might, but this signature scheme is vulnerable and requires additional receiver resources. With many entities out- sourcing email services, messages will typically traverse different acceptance policies. In the case of the TBR extension, content based policies can be consolidated at the last stages. Without reducing email's delivery integrity, the TBR extension also curtails DSNs when the message is never fetched due to the origination being considered abusive or the recipient being invalid.

We don't save cross-net transfers. We add transaction overhead and delay.

The cross-net transfers are not increased when applying point of origination reputation. Due to the prevalence of spam, point of origination reputation (POOR) can significantly reduce the overall network traffic. When blocked with 451 4.7.8/9/10, the TBR extensions provides a means to complete the transaction. When the receiver decides to obtain content is when it becomes obligated to ensure the message is delivered or the MAIL FROM receives a DSN. The TBR extension also ensures the MAIL FROM is valid without demanding any additional analysis by the receiver.

The issue of hand-off responsibility is changed, but I have not heard that asserted as a problem amidst anti-abuse efforts.

This is a problem when out-sourcing email services and often why recipient lists must be continuously updated. This issue is also likely a reason for the tens of millions of servers now issuing DSNs containing the original spam and malware. : (