Re: [Asrg] Spam, why is it still a problem?

On Jan 16, 2006, at 03:25, Craig Cockburn wrote:

> Or more to the point why are we letting it still be a problem?
<snip/>

> Given that spam is still a problem, comprising about 90% of email  
> and costing about $40 billion a year, why is it that there is so  
> little visible progress on this list and generally regards  
> implementing a solution that actually works and which can be easily  
> accessed by the average Internet user? (and even better open source).
Craig,

There are two issues that dominate this problem.

Uncertainty:
While it is sad that one person's spam is another's important  
marketing communication, that uncertainty in categorization requires  
spam filters to conservatively err on allowing spam through.
Spammers exploit the internet engineering prnciple, "Be liberal in  
what you accept, and conservative in what you send”, to their  
advantage. To fix parts of the spam problem, we must become much more  
"conservative" in what we accept. Because email is such a critical  
service, many organizations refuse to tighten up their email  
acceptance policies, thus potentially rejecting important  
correspondence. Little things like following the RFCs and requiring a  
reverse DNS entry for SMTP servers can reduce spam. Another example  
is the "early-talker" filter, this filter rejects connection attempts  
that do not follow the SMTP protocol by waiting between 5 and 20  
seconds before responding with an SMTP greeting. Because the  
malefactor starts speaking too soon, this knocks out that connection  
attempt (15+% of the spam hitting my system). In other words,  
reducing uncertainty by following the rules of SMTP will help combat  
spam.
Finally, independent of which anti-spam validation an organization  
chooses, SPF, DKIM, BATV, BondedSender, etc., the key thing is that  
organizations should choose one or more them. Shining a "technical  
light" on yourself as a reputable emailer really helps battle spam.  
All of the other tools become better when they have a reliable  
identity to depend upon.
In other words, spam exists because of uncertainty in identity and  
ambiguity in interpreting the SMTP RFCs.
Money:
The people who can fix the problem do not suffer (much) from the  
problem.
ISPs, in fact, incur uncompensated support costs when they try to  
impose technical fixes. The cure is more expensive than the disease.  
Therefore, they do not attempt to fix the problem. (Yes, ISP  
employees suffer from spam but they are relatively easy to support  
compared to the general public.) Also, ISPs and ESPs cannot work  
alone. Because if they do, they risk losing customers to competitors.  
In other words, there is no incentive to act incrementally. That is  
why it is so important that AOL, Yahoo, Google and Hotmail need to  
all agree on a few mechanisms they will use to tighten up the SMTP  
infrastructure. Currently, they have not agreed.
ISVs also have weak incentives to solve the spam problem. Why would  
the manufacturer of Outlook, a free email client, fix it for spam? It  
handles mail just fine. Microsoft, in this case but the economics  
apply elsewhere too, will make no money from fixing the client. They  
will make money from selling a license to Exchange. Needless to say,  
they will focus upon Exchange. Anti-spam server software is a value  
added service. (And, due to support costs incurred when upgrading the  
client, the server is the lowest cost place to address the problem.)  
In fact, the longer they wait, the more likely that some solution  
will emerge from the open source shadows or the de jure IETF process.
Anti-spam legislation is typically implemented by consumer affairs  
staff in state Attorney General's offices. As with almost all  
consumer law, enforcement is scaled based upon harm or population of  
harmed citizens. In the scheme of things, spam does not have a large  
harm. Phishing has large economic harm and, hence, you see some  
action there.
Summary:
There are more issues involved than the ones I list above but I think  
they are the most important. Others may have other candidate causes.
Regards,
Andrew


____________________________________
Andrew W. Donoho
awd(_at_)DDG(_dot_)com, PGP Key ID: 0x81D0F250
+1 (512) 453-6652 (o), +1 (512) 750-7596 (m)

"To take no detours from the high road of reason and social  
responsibility."
    -- Marcus Aurelius


_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg