However, you would cut down you spam load significantly if you rejected
all mail with "Content-type: text/html" SMTP or MIME entity headers
from strangers.
How can you have it both ways? You either receive email from strangers
or you don't.
The point is BY DEFAULT to reject unsolicited HTML-burdened E-mail, or mail
from untrusted/unknown senders that contains attachments. By so doing, you
will shrink spam byte volume by probably 90% or more. You'll also eliminate in
one fell swoop the GREAT majority of viruses, worms, and trojans.
Plain ASCII text will continue to be delivered as before.
Those who want to recieve mail from strangers can simply
not block any email, or continue to use filters.
There is nothing that would PRECLUDE anybody from continuing to receive
anything they like, from anybody (including all the world's spammers and other
abusers), if that's the way they want to set their incoming mail account. (In
fact, I'd presume that honeypot accounts would want to be set to receive
anything and everything...)
Those who don't, should be able to require verifiable identification through
some means.
Once you HAVE heard from someone, there are a whole variety of ways to
establish "identification", perhaps the most recognized being PGP/PKI
encryption or signatures. But again, that's not particularly useful for a
FIRST contact, and that's really what we're talking about here.
It's really sorta like Yahoogroups discussion lists, where their enlightened
feature of "moderate posts by new members" has contributed HUGELY to keeping
those lists (the well-managed ones, anyhow) spam-free. In that case, newly
subscribed folks CANNOT send anything to the whole list until the moderator
approves it. Once the moderator is convinced that the subscriber is not a
spammer, the specific now-trusted user can be set to "no moderation required"
(and that status can be reverted to 'moderated' if the trust proves to have
been premature). Meanwhile, hit-and-run spammers who create a new disposable
ID, join a bunch of lists and shovel out a boatload of spam and then vanish
into the dark of night have been severely damaged, now that only the moderators
ever see the great majority of their crap.
What we're talking about doing here is much the same thing. We're trying to
put obstacles into the paths of spammers, denying them much of their cover,
making their lives more difficult in hopes of convincing them to go find
themselves some other way to try to make a living.
If one's bread and butter dependes on receiving email from strangers,
then they need an open mail box.
My proposal would leave virtually ALL mailboxes 'open' (subject, at least, to
whatever content filtering [Spam Assassin or whatever] they wished to employ).
Let's recognize and agree that maintaining the status quo isn't really an
option... spam is a very big problem, and it WILL GET BIGGER until eventually
even the most reluctant will agree that SOMETHING serious must be done.
Blocking (by DEFAULT) unsolicited HTML-burdened E-mails from unknown/untrusted
recipients would probably block 70-80% of today's spam E-mails, and even once
*every* spammer reverts to plain ASCII text, the spam BYTE VOLUME (for the same
number of spams) would probably still be reduced by something like the same
amount.
Yes, it IS true that a lot of clueless users end up sending (usually totally
unwittingly) HTML-burdened E-mails because their AOL/Hotmail/Exchange/whatever
ends up generating that BY DEFAULT. [IMHO, that is every much an inexcusable
waste of resources as spam itself is... but in any case, they certainly have
the option of sending that to anyone willing to agree to accept it.]
Most (I believe a very high percentage) of email users don't depend on email
from strangers for their living.
Okay, let's accept that.
Still, I want to be able to receive inquiries from people who I've never heard
from before... folks maybe who stumbled into my Web site, or who got a referral
from another of my clients, or maybe just someone who saw one of my E-mails
somewhere and wanted to get in touch. [Over the years, I've hosted dozens and
dozens of people from six continents who I've met JUST THIS WAY...] It matters
little whether it's "making a living" or just "making friends". The thing that
is important here is that plain ASCII text (the UNIVERSAL form that EVERYONE
can handle) is _the_ preferred way to E-mail someone, AT LEAST UNTIL YOU HAVE
ESTABLISHED THAT THEY CAN AND WILL ACCEPT OTHER FORMATS.
Sure, once upon a time there were some newbie folks who tended to send their
E-mail as attachments in Word .DOC format attachments too, but thankfully most
of them figured out pretty quickly that they didn't endear themselves to most
of their recipients by doing that. :-) [Now, mind you, there's NOTHING WRONG
with using Word .DOC files if you REALLY WANT to send something that way, and
if your intended recipient(s) agree.]
[snip]
(Could you adjust your line length to less than 78? My ancient MUA
doesn't rewrap lines, and I think it shouldn't. I control my own line
lengths by piping paragraphs through `fmt` as I edit them with `vi`.)
Could, but in my experience the artifacts of doing that tends to be worse
rather than better, since one ends up with chewed-up interlaced paragraphs that
are even more frustrating. Sorry.
[snip]
One nice thing about what I propose is that it can be implemented at the
recipient ISP ...
Yes, but until ISPs can tell users who complain "run the Netscape or
Microsoft auto-update mechanism", they will not and cannot do such
filtering by default.
I'm not sure I understand your comment. The reason they would want to change
THEIR mail software would be so it would NOT generate HTML-burdened E-mail by
default. To truncate the undesirable/untrusted stuff earlier on the
transmission/delivery path (BEFORE wasting as many resources handling it) it
makes sense to filter it as soon as it reaches the destination domain. So
while the filtering COULD be done at the end-user's system (indeed, my personal
incoming mail filtering system works on stuff after it's arrived here) the
benefits are greater if you filter it sooner.
[snip]
How much of that [HTML-burdened] mail involves real
formatting? If you choose 50 messsages at random, how many use HTML
and of those, how many use any real text formating? (e.g. intentional
use of <em> or <b> as opposed to noise added by MUAs to all HTML
messages they create)
If you'll take bets, put my money on 80% or more using HTML and
less than 1% using any real text formating.
In the case of Yahoogroups, and this is probably typical, the main HTML
formatting difference is that it 'allows' Yahoo to install bulkier, more
annoying graphical banner ads in messages instead of just sending (smaller)
plain ASCII text ads.
I don't think that most Lusers do any message formatting in their messages...
hell, many of them haven't even figured out how to work their shift and caps
lock keys. ;-) And even those that DO would probably gladly give up that
*minor* characteristic if the result was a major reduction in spam.
So turn off HTML until the the user makes some text bold or purple,
and then silently and automatically switch to HTML.
[Or, pop up one of those dialog boxes alerting them that HTML-burdened mail
adds 3-5x to mail bulk and isn't welcomed by all recipients, and allowing them
the option of proceeding or going back to plain ASCII text... with maybe an
option box they can check which says "don't warn me about this in the future."]
As I keep saying there are those of us whose livelihoods involve
email and who won't tolerate 0.2% false positives and there are
the 100,000,000's of the rest of the Internet for whom not receiving
spam is more important than not hearing from a long lost friend.
And let's not forget that the "long lost friend" probably won't object in the
slightest to sending a plain ASCII E-mail, if that's what it takes to reach
you. I don't know many friends who would begrudge anything that takes a
substantial bite out of spam and viruses!!
Without HTML
tricks to hide various things, all of their stuff is in the open and
plainly visible to your Sally End Users. Even your Sally probably
discards a Nigerian 419 spam after reading the first 5 words and she
may delete unread mail with obvious "hash busters" in subjects.
Exactly.
FWIW, I end up opening and reading those mails mostly so I can add the bad
guys' URLs and domain names to my PC's HOSTS file, which serves as one of the
inputs to my incoming mail filtering system. :-) I'll therefore never be
bothered by reading spam hawking THOSE sites ever again. :-)))
[DCC]
That idea is that as a message
arrives or is about to be presented to you, your MTA or MUA should ask
a clearinghouse if substantially identical messages have been seen
elsewhere. If the answer is "yes, 12345 of them," then your MTA or MUA
can know it is a "bulk" message check your list of approved senders or
whitelist.
One of the perhaps useful ideas from the spf discussion is that rather than
itself being a filtering determinator, maybe the spf analysis (i.e. 'is this
sender expected to be coming from this IP address block?') would be one,
weighted input that could be included in the SpamAssassin criteria list.
By the same token, knowing that a lot of folks received the same message (or
substantively the same message) from your inbox that is being examined would be
another useful weightable input into the SpamAssassin (or similar) decision
process.
[snip]
IMHO, blocking an email message because it's base64 encoded is
misguided. Here's a case in point: There are gateways out there that
will switch encodings on messages. If a message is sent through a
gateway that understands 8-bit messaging, and it is then passed through
to another system which only allows 7-bit, it's got to be downgraded.
Downgrading to base64 is perfectly reasonable for such a gateway.
While I understand your point, I think I have seen EXACTLY ONE such message
that I can recall (and THAT one was spam, where the encoding was used just to
obfuscate the mail content), and I suspect that the number of remaining
7-bit-only MTAs is exceedingly small.
But again, in any case, if a given user tends to receive mail (legitimately) in
base64 format (perhaps from a specific subgroup of correspondents) it would be
very easy to open that window for those folks (and even, presumably, to disable
the encoded-text filter altogether for that recipient). Again, it's about the
CHOICE for the user, and so they could respond in whatever way and degree makes
the mose sense TO THEM in exchange for a given degree of control against
incoming spam, viruses, worms, and trojans.
[snip]
[HTML-burdened E-mail]
This issue is well documented in <http://www.camblab.com/nugget/htmlmail.pdf>
Indeed, I've bookmarked that for future use. [the article makes a good
argument against HTML-burdened E-mail, and describes how to send plain ASCII
text for various mail client software]
[snip]
[<http://www.camblab.com/nugget/htmlmail.pdf>]
Interesting read. However, how can we mandate filtering of HTML mail if
many MUAs use it by default?
I don't think we can "mandate" on a global basis much of anything. (Part of
why I think approaches like spf are probably doomed). What we *can* do however
is to make antispam tools like my proposed "permission whitelists" available,
and encourage people to use them as widely as possible.
Indeed, such an approach is arguably at its MOST effective when it is NOT
universally used, and where nearly all (offending!) spam will be blocked by it.
:-))))
[Although, as I've pointed out, even if ALL spam reverts to plain ASCII text as
a result of (someday) near-universal HTML-burdened-mail blocking by default
from unknown/unagreed senders... you still have the volume of that spam reduced
by at least 70%, even for the SAME number of spam messages... and probably a
greater reduction due to the resulting plain ASCII text spams having a harder
time trying to trick users into responding, making the whole enterprise less
appealing to spammers...]
Obviously, the MUAs and ISPs which generate wasteful HTML-burdened E-mail by
default are a BIG contributor to the waste and abuse problem... probably nearly
as bad as spammers themselves are. One of the unabashed goals ought to be to
get such mail sent in plain ASCII text instead (I liked the idea of sending it
in plain ASCII text **unless** the sender *actually used* features requiring
formatting... and perhaps answered a dialog box confirming that such formatting
was important to them, and agreed by their recipient(s).)
[snip]
Basically we are in an educational period where we have to undo the
damage done by some bad decisions by Netscape and MS. The consequences
are now apparent re HTML. Even MS realizes the error of its ways (well,
some of them anyway) and is now turning off some default security
threats in its new builds.
It would be interesting to see a suit filed against AOL and/or Microsoft by
some big ISP (say, swbell.net or earthlink or somebody?) to attempt to recoup
the heavy costs incurred (bandwidth, storage, etc) due to the wasteful default
sending of HTML-burdened E-mail where it proves to not be necessary. Perhaps
that's just a dream, but I'd love to see them held accountable someday for the
huge associated costs that ROTTEN decision carried.
[snip]
The vast majority of end-user email is HTML. They use bold and italics in
all
their other applications. They embed cutesy images in all their
other applications. Why shouldn't they in email? Cost differential?
They really don't care.
And they shouldn't HAVE to care, as long as they have permission from their
intended recipients to receive such stuff. But SOMEONE pays for all this
bandwidth, and a lot of it is wasted by just such irresponsibility.
If we agree that spam is wasteful, then we ought to agree that
unnecessary/unwanted/[unwitting!] use of HTML-burdened E-mail is also wasteful
(and HTML-burdened spams just COMPOUND the problem to a higher degree!)
Anything we can do to encourage stopping that waste is probably a good effort.
And then of course there is commercial, requested email, which is virtually
100% HTML.
And such commercial, requested, HTML-burdened E-mail... which the intended
recipient has AGREED to accept... will not be affected at all by the
permissions list, since it will (obviously) be approved by the recipient in
advance.
Again, the whole point of this is PUTTING THE RECIPIENT IN CONTROL of the use
of their LIMITED RESOURCES (bandwidth, Inbox space, storage/archiving
resources, etc). THAT IS AS IT SHOULD BE! The phone company has the right to
publish any kind of phone books they want (as long as it's at THEIR expense),
but they don't have the right to bring by a truck and dump 30,000 of them onto
MY driveway!
[snip]
And if they REALLY WANT to use HTML, another approach without most of the
bulk is to put it up on a Web server (using an 'unguessable' URL) and E-mail
(as
plain ASCII text) the URL to the intended recipient.
Sure. I'll just tell my father that when he wants to emphasize
something he shouldn't hit Apple-B,
If he wants to emphasize something, it's just as easy to *emphasize* it or
_underline_ it or CAPITALIZE it. This has been standard practice in e-mail
messaging for something like twenty years, and you don't have to triple or
quadruple the size of the overall E-mail to do that.
he should pay his ISP for a web host and learn how to publish on it.
1) I don't think that most end users really do need anything that elaborate.
2) Most end users ALREADY have (at least some, but usually more than adequate)
free personal web space already made available to them by their ISPs.
3) Suitable software could make that publishing-and-linking 'transparent' (or
nearly so).
4) If they REALLY want formatting in their text, then there are other ways FAR
better for doing that than HTML (.DOC files, .PDF files, LaTeX files, etc).
But IN ANY CASE sending those is NOT appropriate or polite on a "first
contact", unsolicited basis.
What I was referring to was more for those commercial things (newsletters for
example) where the senders REALLY REALLY FEEL that they "need" HTML to get
their message across... and where their recipients agree. You STILL don't need
to put it into every E-mail you send, you can just send the URL and put the
HTML up on a Web server somewhere. HTML was created for Web pages, that's
REALLY where it belongs... not in E-mail.
NOT EVERYBODY reads their E-mail using a Web browser.
Ask people with text pagers (for example) how much they appreciate receiving
E-mails where the only thing they see before the message truncates is a bunch
of stupid HTML tags...
Or were you instead proposing that everyone in the world upgrade their email
client to something that uses "pull" instead of "push"? Please can we be
serious?
I'm being PLENTY serious... you are the one talking about silly stuff.
[snip]
...in the business world...they use Outlook, which gives you a little toolbar
that looks exactly like your Microsoft Word formatting toolbar, with font
size,
bold/italic/underline, colour, etc.
Fine. Doesn't change the situation much. Fact remains that the great majority
of E-mails (even the stuff sent with HTML-burdened format) does not include any
use of font sizes, bold/italic/underline, colors, etc. And if people realized
that doing so inconvenienced many recipients and increased the cost of their
E-mail (and specifically, if it put a serious dent in spam!) they'd find other
ways to get their message across.
If 70% of people currently use HTML, as Kee Hinckley's numbers suggest, then
the false positive rate will be 70% under the proposed scheme.
You're ignoring the facts that:
1) it could be that existing correspondents will be automatically,
semi-automatically, (or manually!) set to continue authorizing them to send
mail in the same format they're used to sending it;
2) hopefully, once they become sensitized to the wastefulness, they'll be
glad to send plain ASCII text instead;
3) if it means combatting spam, they'll be even more happy to adjust their
settings;
4) hopefully ISPs and mail client authors will also become more responsible
and change their defaults to something less costly and less spam-friendly.
Think of what would happen in those 70% of cases.
After the initial installation, that's only for the "70%" of NEW e-mails
arriving in unwanted formats from unknown/untrusted senders. (And, presumably,
90-95% of all spam!)
Instead of one message, there would be at least three (the original HTML
mail, the bounce message, and the plain-text version),
The bounce message will probably be VERY small (no need for IT to be
HTML-burdened! :-) ) and the plain-text version likewise. Hopefully in very
short order, people will figure it all out.
and at most five (the original HTML mail, the
bounce message, the request for permission, the acknowledgement of
permission, and then the original HTML mail again).
In the GREAT majority of cases, there won't be any need for permission.
For a proposal that's meant to decrease the volume of mail and avoid creating
disruption, that's not very satisfactory.
Either senders switch to sending plain ASCII text, or they don't.
If they DO, then the mail volume (bulk/bytes) goes down by about 70-80%
starting with the VERY FIRST plain ASCII text E-mail they send. It doesn't
take very many of those before a couple of explanatory/permission messages
repay themselves.
If they DON'T, then at least they'll become aware that many users resent the
waste of their limited Inbox space (and users will hopefully learn the
downside, which they generally don't understand today). And they'll learn to
not PRESUME they can send "just any" formaat that they wish without having the
RECIPIENT okay that.
In either case, the clear message is that the RECIPIENT HAS THE CONTROL over
what they are willing to accept... the sender cannot just send 'whatever' and
expect to have it delivered regardless.
Mail from strangers isn't just from long-lost friends. It's ubiquitous in
the corporate world.
Absolutely. And many corporations may choose to accept mail in any and all
formats, and not to implement any kind of filtering at all. That's THEIR
CHOICE.
Given what viruses/worms/trojans (AND spams!) cost most companies, however, I
sorta wonder how many of them will end up making that choice. :-)
The employees of a typical company communicate with employees of many other
companies: clients, suppliers, etc. Employees come and go, and different
projects involve different combinations of companies and different people,
so the set of people that a given individual may receive legitimate mail
from is constantly changing. Consider a sales representative, who may be
contacted by a constantly-changing set of hundreds of people who work at
companies that she is negotiating sales with, or that may be interested in
her company's products. As a sale progresses, more companies (accountants,
lawyers, etc.) are brought in. Similarly for mergers and acquisitions.
That's a lot of people who would be annoyed by bounces.
Right. But I suspect that the great majority of those senders will pretty
quickly realize that they won't be inconvenienced if they send plain ASCII, and
make the switch. It's a (minor) learning process, but then again SO ARE ANY of
these proposed anti-spam solutions. Maintaining the status quo is NOT a
practical and viable option.
[snip]
On receiving your proposed bounce message, the irate recipient gets on the
phone to his IT department, and demands to know why he has to go off and read
some technical web site just to send an email. This is a recipe for chaos.
The bounce message ought to be polite and informative, and explain why we all
need to work together to stop spammers and viruses. It's true that some users
will call their IT departments instead of visiting the Web site for more
information, but as I mentioned before, it's a learning process. As such
"permissions lists" become more common around the Net, legitimate senders will
learn to change their outgoing mail options and everything should quite rapidly
settle down.
And in case you're thinking of suggesting that the IT department configure
everyone's MUA to send plain text by default, keep in mind that this would
place a huge burden on IT departments,
Oh, that's SUCH nonsense. It's probably a SINGLE registry setting, and many
large-corporate IT departments probably already have "automated rollout"
provisions to adjust stuff like that. Even if they don't, it would be a VERY
easy matter to have a 'stock' E-mail that explains to the user how to adjust
their settings in Outlook accordingly. "Huge burden" is ridiculous, especially
if there was in fact a great reduction in spamming, viruses, worms and
trojans... which DO cause a "huge burden" on corporate IT departments.
which are already straining to keep everyone's virus checkers up to date.
Virus checkers, in particular, would become *almost* superfluous if attachments
were only received from known/trusted senders WITH A RECOGNIZED NEED TO SEND
ATTACHMENTS!
Moreover, it wouldn't help in the case where the irate manager is trying to
send a PowerPoint document.
There's NO point in sending a PowerPoint document to someone who doesn't have
the requisite software to view a PowerPoint document. Again, there's NO point
in sending such stuff to anybody on a FIRST CONTACT.
An anti-spam system should be as non-intrusive as possible;
Well, other than that we sorta wanna rock the spammers' world. :-)
...I think we can do better than this.
Perhaps, although this is the most practical, easily implementable, functional,
compatible, lowest cost proposal I've seen so far. And about the ONLY one
which would probably cut overall E-mail cost and volume by half or more,
Net-wide.
[snip]
More important, the problem for your idea is not selling it here but
in Redmond. Until Microsoft stops encoding mail in HTML by default,
perhaps by only turning on HTML when the user does some formating,
ISPs cannot filter HTML by default. I really wish they could and
would, but I'm not quite crazy enough to confuse my wishes with reality.
If Microsoft can sue a bunch of spammers for generating
wasteful/fraudulent/bogus traffic and increasing Hotmail's costs, then a
similar argument could be made for a big ISP (say Earthlink or someone) that
Microsoft's defaulting to sending HTML-burdened mail is similarly costly to the
ISP. While we haven't seen such a suit yet, there are a whole lot of lawsuits
based on much slimmer and more farfetched arguments. It would certainly put
the concern onto Microsoft's radar screen, anyhow.
Gordon Peterson http://personal.terabites.com/
1977-2002 Twenty-fifth anniversary year of Local Area Networking!
Support the Anti-SPAM Amendment! Join at http://www.cauce.org/
12/19/98: Partisan Republicans scornfully ignore the voters they "represent".
12/09/00: the date the Republican Party took down democracy in America.
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg