because the "large numbers" of mail will be
significantly smaller than the "massive numbers" we're seeing today.
I don't think that's assured.
Please justify this statement. I've taken some numbers provided by a
poster, and indicated the difference in the quantity of resources
available - even assuming the spammers can use a significantly larger
fraction of their zombie pool to make stamps than send mail. I'll even
summarise here:
At present, spammers use fresh zombies for an average of 4 days after
infection - this pool is about 100,000 machines at present.
The total zombie pool is about 1,500,000 machines at present, depending
who you ask and how they collected their data. This is about 15x the
number of active spewers.
Assumption: It presently takes about 4000 times as long for a typical
zombie (1.0GHz P6 or K7 class CPU with a 128Kbit DSL uplink) to mint a
20-bit hashcash stamp as to send out an unstamped, 5KB mail.
Given that spammers can recruit the entire horsepower of their zombie
army, compared to only a fraction of the bandwidth, this still marks a
reduction in overall spamming capacity of about 200 times. And this
assumes the zombies' CPUs aren't being used for anything else, such as
games.
1) The reason the zombie pool is "only" at 1.5 million machines is largely
because that number is largely sufficient for spammers needs. If they needed
more, they'd infect/recruit more.
2) The zombie pool is clearly not cranking out spam at anything like its
potential capacity... even the last four days worth of infected machines are
probably not producing spam at anything like their total capability.
I really don't believe that the solution to the spam problem involves
making
E-mail harder or more costly to send. I think a much better solution
involves
making it (if unwanted) *far* less likely to actually be delivered...
such that
the spammer realizes that the effort is probably futile.
Likewise, we HAVE to work on cutting down the ease of zombie
recruitment, and I
believe that my attachments (and HTML!) permissions list idea
(basically a
fine-resolution whitelist) is a *major* step in the right direction
there.
How is your proposal incompatible with mine? I can't see any
particular way.
It doesn't HAVE to be mutually exclusive to make hashcash a bad idea.
Just for reference, I turned up the score for MICROSOFT_EXECUTABLE in
my SpamAssassin filter long ago. Now I don't get any of the usual
viruses,
And that's fine, but it goes back to the not-fine-grained approach that my
proposal specifically doesn't use... where these values would be individually
specifiable by a given recipient for EACH SENDER. I *do* happen to
occasionally
want to receive executable attachments... it's just that they're from a VERY
small percentage of my correspondents, and the receipt of the executable
attachments is NEVER unexpected.
but I get the encrypted ZIP archives instead. I'm sure
SpamAssassin could easily be modified to handle everything you propose
Sure, as could any kind of good E-mail filter. SpamAssassin is fine,
functionally speaking (it would have been (probably MUCH) better if it had been
written in SPITBOL, but that's another story). SpamAssassin is a good step in
the right direction (and perhaps the best thing of its sort out there today).
I
don't happen to think it's good enough, AND I think that most of the protection
against spams/viruses/worms, to be meaningful and effective worldwide, needs to
be BY DEFAULT... thus built in either to the OS, or the mail client software
(say, Outlook/Outlook Express/etc) that "most" people use. And not an
independent addon program, which is unlikely to be widely used enough to be as
effective as this kind of thing wants to be. (My approach DOES _immediately_
benefit those adopting it, but in order to really make a difference in spam we
see on the net worldwide, a lot of people should be using it.)
Clearly, though, you're right in that SpamAssassin needs to be able to treat
ZIP
archives (and with or without encryption) as "dangerous" and as one of the
available filter criteria.
- so why don't you talk to them instead of shooting me down all the
time?
The people who I'd *like* to see[/help them] solve the problem would be
Microsoft, since if they solve it at their level then truly a *large* number of
folks around the world will benefit from the improvement.
As for shooting you down all the time, please don't take it personally. There
are a lot of people pushing for a lot of harebrained and ultimately
not-very-clever "solutions" for these problems, and most of the ideas simply
aren't very good. Before we go off halfcocked racing down a path to a future
that isn't very well thought out, we need to ALL consider these proposals and
EVERYONE try to foresee what's wrong with them. Otherwise, we're going to go
way far down a road towards a costly, unworkable "solution" which is going to
leave everybody wondering "What the hell were those guys thinking?"
We should be able to do better than that.
Since when is that such a weak argument?
Well, maybe always, in part since it ignores the very legitimate needs
of
organizations like Yahoogroups and other major outbound mailers to
send millions
and millions of pieces of mail a day.
Did I say that every piece of mail needed a 20-bit hashcash stamp? No,
I said that whitelisted senders get to use smaller, even
computationally trivial stamps. Subscribing to a mailing list should
include making a whitelist entry at each end of the transaction.
I don't like that. First, because I don't think the sender should have to do
ANYTHING different on a recipient-by-recipient basis (other than, perhaps,
deciding who will tolerate bulky HTML-burdened E-mail (that's if the sender
REALLY wants that kind, rather than just using ASCII) and who wants plain ASCII
text instead...) I don't like two-ended solutions, since it invariably seems
to
involve coordinating things that end up being hard to coordinate, and ends up
forcing mutual concurrence on things where it's often hard to get agreement.
Secondly, it means that the recipient has to whitelist a *lot* more of the
senders that they get mail from than they should have to whitelist. If you
have
a reasonable policy for default mail permissions, then most of the senders you
get mail from (even unexpectedly) shouldn't be affected by your filter. The
goal of my scheme is a fine-grained sieve putting the RECIPIENT in control, and
which lets as much of your legitimate mail as possible through but which turns
aside virtually all the spam, and without significant negative implications on
legitimate and responsible senders.
And, all this without having to change the basic worldwide Internet E-mail
infrastructure, or significantly increasing the fixed/incremental cost of that
infrastructure.
I also said, at some point, that if a message is addressed to a mailing
list and the stamp matches the mailing list's address, then if the
recipient is subscribed to the mailing list he should accept the stamp
as valid. This caters for listserv software that doesn't (yet)
understand stamping, signatures and whitelists.
But you presume that everyone SENDING to that mailing list is going to go along
with your "stamp" and signatures proposal and deal with the hassles, right?
Again, I don't much like proposals that force EVERYBODY around the world to
change their behavior. And it's bad enough when it's TWO people that e-mail
each other directly; it's much worse when you have third parties (like
Yahoogroups) involved and you end up creating this kind of great scism between
communities.
Here's my point, since you seem to have missed it up until now:
Hashcash and signatures work in concert! One covers some situations,
the other covers the rest. Please stop taking them in isolation to
make straw men.
Both of them require components to be installed EVERYWHERE, both to generate
them and to check for them. I don't object to multifaceted solutions (hey, I
specifically endorsed a SpamAssassin-like content filter working in conjunction
with my permissions lists.)
But if I don't like attempts to increase the cost of mailing by hashcash, and
don't like cumbersome "signature" or "certificate" approaches, I don't know why
you'd imagine that I'd like a solution better that involves the combined
disadvantages of EACH of those.
What determines whether it's spam or not is whether the RECIPIENT
wants it and authorized the sender to send it to them, and whether the
content of the mail is remotely like what the recipient(s) expect and
want to receive from that sender.
Content filters only work to a point. SpamAssassin actually has
filters set up to detect most content-disguise mechanisms already -
stuff like an excessive number of HTML tags, FONT COLOR="white", etc.
Spam *still* gets past it occasionally.
Sure. But in conjunction with my permissions lists, there will be many fewer
ways to deceive and bypass it, while still getting the things through that you
occasionally need to receive from SOME selected few of your correspondents.
Presently, setting your filter settings tight enough means that you're
filtering
out some stuff that OCCASIONALLY you need and want to receive from SOME folks.
I don't suggest that SpamAssassin is perfect. It *looks* pretty good. It
would
be *better* if it didn't have quite such a difficult job to try to do (and my
permissions list makes its job a LOT less daunting). If SpamAssassin were
written in SPITBOL, they'd have far more powerful pattern recognition
capabilities available to them at the core programming-language-level, and it
would be a lot harder for spam to slip past it.
But I also don't think that the spam filter HAS to be absolutely 100% effective.
It's probably enough that the spam problem is reduced to an occasional nuisance.
It's perhaps true, though, that by the time we t-can that much of the spam,
combined with making it MUCH MUCH harder to recruit spambot zombies, that a lot
of the spammers will simply go look for more promising ways to pursue their
scams.
We haven't seen the worst of spam, yet. But eventually the tide will crest and
we'll start seeing an improvement.
As for whether the recipient authorised the sender... without any
widely deployed authentication mechanism, how do you avoid spoofing on
the whitelist entries?
Spoofing only helps if the permissions assigned by a given recipient for the
sender you're spoofing helps you send the stuff you're (as a spammer) trying to
send. In general, you simply don't know that it does... you don't know WHO you
want to spoof, and you don't know WHO has assigned them the "advanced"
permissions that apparently you're hoping to benefit from. If ultimately the
majority of recipients have VERY few people with advanced "whitelisted"
permissions, then your odds of hitting one at random are very small.
Why do you think that it's not enough to simply use the E-mail
characteristics
(type of content, HTML classes used, type of attachments, size maybe)
in
combination with the sender and the list of permissions granted to
that sender
by the recipient?
Because we already have that general type of filtering, and it isn't
working well enough.
No.
1) We don't have that on the basis of a sender/recipient pairing;
2) We don't have the kind of fine-grained control based on the types of HTML
codes present and the types of attachments in use;
3) Gratuitous HTML provides a lot of underbrush for spammers and abusers to
hide in. Far too many people presume that they can send HTML mail and that it
will be unquestioningly accepted by "everybody".
4) Far too few people have even the "general" kind of filtering that you're
talking about implemented.
Even if *I* implement completely all of my ideas, and use them here, the fact
that they only help ME is still going to run afoul of your "it isn't working
well enough" condemnation.
There are a variety of commonly recurring types of spam that still make it
through my present incoming mail filter. In part that's simply because it's
small enough in volume (so far) that I haven't busied myself to make the
appropriate improvements to the filter to detect and t-can those types.
And if you DO want something more, why is a "passcode" or
something freely made available by the recipient (perhaps in large
number of
variant forms, so to make life difficult for address harvesters) and
included in
(say) the Subject or a special header line perhaps, not enough?
Because that's too much effort for most users - it's "intrusive". I
have seen people using it, but they're distinctly in the minority.
It's certainly less intrusive than having to buy E-postage or computing
"hashcash". And this would work with basicaly all existing E-mail programs.
I do agree though that I like single-ended solutions better in principle.
(I'll
point out that, unless I'm mistaken, NONE of the other solutions we've been
discussing here are single-ended, other than my permissions list idea).
Each signature, incidentally, will be valuable only for the number of
recipients who have whitelisted it, which for Joe User will normally
be quite a small window.
So what happens if they HAVE NOT whitelisted it? I think that many
(most if not
all!) types of users NEED to have SOME kind of way to receive
legitimate,
unanticipated E-mails...
Then they mint hashcash for it. It only takes a few seconds of CPU
time for most people. If they happen not to have the CPU resources
available to do it themselves, they have the option of paying a third
party to do it for them.
Again, complicated and it increases the cost, while not solving the problem.
Besides, anything keyed to "consuming processor time" is inherently a decaying
thing, as processor speeds increase.
The signature is merely a way to bypass the CPU expenditure for regular
correspondents - friends, family, regular business contacts, mailing
list subscriptions, etc. Nothing more.
Signatures require a concurrence at both ends on the type of signature to be
employed, and much existing E-mail software (including embedded apps that
generate and send E-mails) obviously don't generate mail containing them.
Again, you're talking about changing the world's E-mail infrastructure.
Possible, maybe, eventually, but not right away and not without a lot of grief
in the meanwhile.
Who really cares about signing keys, or certificates, or all these
SPF-like mods
to DNS which don't solve the problem, or E-postage and all the new
opportunities
for fraud, abuse, extra costs and hassles? I sure don't. All I'm
concerned
about is dumping spam in the trash so I don't have to deal with it.
Then implement your filter for yourself.
Fine, and I've done much of that. As I said, I continue to refine my filter...
I'm currently up to revision 1.43 (since 2002 when I first wrote and started
using it) and in general it works pretty well. The present filter is 1742
lines
of (fairly heavily commented) 32-bit SPITBOL source code. (It does a LOT more
than just antispam filtering). It works in conjunction with a couple of
"helper" apps and a variety of data files (including dictionaries) and produces
a variety of log and audit trail files. The current version was written with
multiuser features to support enterprise use, although nobody except me is
presently using it on a daily basis.
It sounds like something you
can implement and gain benefits from on an individual basis,
Can, have, and do! :-)
What I presently have here doesn't fully implement my proposal, but that's
because it's presently "close enough" to solving enough of my own personal spam
problem that I don't feel a pressing need to implement the rest (not for
myself,
anyhow).
The framework I have would probably support all of the things I'm talking about.
so you
shouldn't really have any trouble with it. If it works for you, come
back and tell us about your experience.
That's largely why I'm here. :-) I've been observing the problem for a long
time, and particularly in the context of seeing what I already have and what
things one could do to pick up on the remaining "problem" spams.
But don't just pan everyone else's ideas while your own is still theory.
I think that:
1) My ideas are a LOT more than just "theory".
2) My ideas hardly require any kind of dramatic technological breakthroughs
that would call into question their implementability.
In fact, of the ideas being discussed here, mine involve a whole let less
"waving of hands" and "blue-skying" than hashcash, OR E-postage, OR signatures,
OR certificates, or global changes to the Internet's DNS infrastructure, or
indeed just about any of the other ideas that we're discussing.
So I think it's pretty silly to condemn anybody's ideas here as "just theory".
By definition, this is an "asRg" and the R stands for RESEARCH!
Gordon Peterson http://personal.terabites.com/
1977-2002 Twenty-fifth anniversary year of Local Area Networking!
Support free and fair US elections! http://stickers.defend-democracy.org
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg