ietf-asrg
[Top] [All Lists]

RE: [Asrg] The "Human-Shield" effect; the need for end-user control

2003-03-22 17:49:08
" broadband ISPs in Canada surcharge their customers for bandwidth in
excess of a monthly quota".

1 movie at 10 GB is the same as how many spam messages? Given that 100GB
will soon be the norm...

... why do people think that spam is equivalent to the end of the world
and that  we must give up free speech and be complete trackable?

The real danger on this list is that it attracts those who want to save
us from something and think it is the most important thing. I joined
because I believe we can do better with email and that spam is simply a
side-effect of simplistic protocols and, for now, I do deal with it on
the client side. ISPs may also choose to provide services for their
users.

But, relax, it's not the end of the world. 

-----Original Message-----
From: asrg-admin(_at_)ietf(_dot_)org [mailto:asrg-admin(_at_)ietf(_dot_)org] On 
Behalf Of
Walter Dnes
Sent: Saturday, March 22, 2003 18:24
To: ASRG list
Subject: [Asrg] The "Human-Shield" effect; the need for end-user control

Hello;

  I've reviewed the past few days' traffic, and haven't seen this item
mentioned (maybe I overlooked it).  ISPs can block the most obnoxious
and egregious spammers *ON AN ISP-WIDE BASIS*, and all their customers
are happy.  However, outfits like Topica and Shagmail, that run
unconfirmed lists are a much more difficult problem.  The basic reason
is that their lists (or their customers' lists) do contain *SOME*
willing subscribers.  Let's pick an example number of 10%.  Let's also
assume random distribution.

  An ISP wants to block their biggest sources of spam.  Block 1 source,
and 90% of your customers are happier, because they see less spam.  10%
lose emails they want.  Oops.  Block a second spamsource.  81% of your
customers are happier (less spam), 19% lose emails they want.  Etc, etc.
Keep this up, and you will eventually alienate most of your customers on
a piecemeal basis.  ISPs don't want to risk this, so they don't block
any but the most obnoxious spammers.  The big mailers and/or their
customers know about this "Human Shield" effect.  And they probably
have a good idea of the actual "pain threshold" at which ISPs will start
blocking over this.  Because it's 100% identical content, coming from
the same IP address, no ISP-wide filtering algorithm can possibly be
100% accurate.  So the spammers can get away with diluting their lists,
and bump up the head count for advertising purposes quite a bit without
being blocked.  Blocking countries which effectively mailbomb your
customers carries the additional risk of being called racism in today's
politically correct climate, in addition to alienating the usual small
number of your customers who might have friends/family/business contacts
in South Korea, Nigeria, China, and Taiwan.

  The obvious solution to this is to give end-users the final say on a
per-user basis.  This is usually done after-the-fact.  It has drawbacks
which all stem from the fact that the entire email has to be received...

  - broadband ISPs in Canada surcharge their customers for bandwidth in
    excess of a monthly quota.  I'm not into P2P, so I never get near my
    quota, however, for anyone who is already at or above their quota,
    receiving the body of the spam email costs real money.

  - files have to be stored somewhere.  This means more and/or bigger
    hard drives.  The resulting higher costs are ultimately passed on to
    the consumer.

  - filtering on DNSbls is difficult, and filtering on the user's choice
    of DNSbls even more so

  - filtering on lack of "in-addr.arpa" rDNS, a very good spamsign, is
    also difficult

  End-user-control at the SMTP stage is actually possible, but it's a
relatively new thing.  I'm a customer of a small ISP that runs a
modified qmail, which parses a filter configuration file in the user's
home directory just after the RCPT: stage of the SMTP transaction.  If
the filter indicates the email should be rejected, qmail sends a 550
reject before the DATA: stage begins.

  43 spams in 10 days from South Korea in Korean character set that I
can't read (and my computer can't display) at the beginning of November
2002 is now just a bad memory.  Ditto for the sob-stories allegedly
from Mrs. Miriam Booga, widow of the late dictator of Nigeria, General
Ooga Booga.  Thanks to the zz.countries.nerd.dk zone, South Korea,
Nigeria, China, and Taiwan have disappeared from my email universe.
Blocking the 200.0.0.0/8 CIDR blocks the latest spammer attack route of
open proxies and/or willing hosts in South America.

  The USA has its share of spams coming from it.  Between...
  - Osirusoft which also offers other zones like Spamaus.org's SBL zone
  - open-proxy zones
  - dynamic IP zones
  - a few CIDRs that I've hard-coded in
  - blocking MTAs with no rDNS
..the torrent of US-originated spam has dropped to a trickle.

  In addition to me being a happy customer, the ISP has not alienated
any of its other customers by blocking email addressed to them.  And
the ISP would obviously have less liklihood of being hit by frivolous
lawsuits like Cyberpromo (Sanford Wallace) demanding that ISP-wide
filters be taken down at AOL.

  There are a few drawbacks...
  - this will obviously *NOT* work with multi-hop MTAs that have the
    external-facing machine blindly accept all email and relay it to a
    second machine that does the final delivery.
  - spammers are persistent.  If you block them at your primary MX,
    they'll try secondaries.  Some spammers try secondary MX *BEFORE*
    trying the primary.  Either the backups are identically configured,
    or else you have to run with only one MX.
  - there are questions about scalability when it comes to spawning a
    process that does multiple DNSbl lookups for each incoming email.
    This is less of a problem on *nix-based systems, where unnecessary
    processes can be shut down by sysadmins; even excluded in a
    custom-compiled kernel if necessary.  Windows-based MTAs have
    overhead from a gratuitous gui and a whole bunch of processes that
    aren't really necessary on a dedicated MTA.  And you can *NOT*
    modify/re-compile the "Windows kernel".
  - instead of a sysadmin configuring a potentially complex filter, the
    average end-user has to configure it.  Imagine your Aunt Ethel...
    - logging in to a shell.
    - parsing email headers to determine the originating IP address.
    - running whois to get more info about the spammer.
    - based on the output from whois, deciding which CIDR to block.
    - editting the filter config file with vi to implement the block.
    menu-driven configurators can do basic stuff, but getting the most
    out of the system requires getting one's hands dirty editing the
    config file.
  - since I'm a customer, not an admin, I don't have access to the log
    files.  The ISP emails me a listing of rejects on a monthly basis.

-- 
Walter Dnes <waltdnes(_at_)waltdnes(_dot_)org>
An infinite number of monkeys pounding away on keyboards will
eventually produce a report showing that Windows is more secure,
and has a lower TCO, than linux.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg

_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg