|
Re: Authentication/Reputation, TBR (was Everyone Greylists...)
2007-12-12 18:01:29
On Dec 12, 2007, at 11:42 AM, David MacQuigg wrote:
Hiding valid email addresses is a fundamentally insoluble
problem. If legitimate senders can find out when an address is
invalid (which they must if we are to preserve reliability), then
spammers can do the same. Perhaps we can sacrifice some
reliability for a little more security, e.g. by sending "nosuch
recipient" rejects only when the sender is authenticated and
reputable. The rest can be tempfailed until they give up.
This is solvable. TBR can both limit discovery of valid email-
addresses AND greatly improve delivery integrity.
Section 3.2 of the TBR draft says '''
If no valid RCPT TO address is
supplied, the TBR command will simply fail. If at least one valid
RCPT TO address is supplied, then the TBR eXAM-URI argument will be
accepted.'''
Doesn't this give the sender the same information as a "nosuch
recipient" SMTP reject?
It is fairly common (although fairly problematic) for inbound MTAs to
use different "valid recipient lists" as messages are forwarded to
their destination. With TBR, an MTA may accept any local-part for a
domain, where subsequent MTAs are then able to silently eliminate
recipients subsequently considered invalid. Expunging recipient TBR
references does not mandate DSNs to maintain delivery integrity, since
fetching a message represents acceptance of an obligation to ensure
delivery or report failure.
Providers of domain names, IP addresses, or certificates have a
conflict of interest, and are unable to prevent access to spammers.
Unwilling, not unable.
Unwilling, and likely unwise. There remains an inherent conflict
of interest. Using registration to limit access may also lead to
forms of extortion. What type of non-repudiation protects innocent
domains? Must all messages be signed by a CA? What limits the
number of certificates issued? Should all certificates be aged 30
days before being accepted?
I'm no expert here, but it seems that all of these would be
effective if we had a good reputation system in place.
A reputation system is reactive. Abuse tactics are able cycle within
minutes where reputation would be much after this activity.
Then we wouldn't need high security, just enough to raise the cost
of abusable IDs above the profit from one short spam run. I'm
guessing that profit is somewhere around $100.
Should Gmail charge $100 for their email service? Why expect $100
charges against a stolen credit cards will limit abuse?
A corporate registration would do it, or maybe a valid credit card
with no reported theft in 30 days, or even just a Paypal account.
Domain registrars are not willing to hold requests for even 24 hours.
Any credit card or online account can be compromised. As it is now,
bad actors are able to publish their domain 24 hours before even
cooperative registries indicate which domains are new. The domain
registry process demonstrates the inherent conflict of interest with
providing access and preventing abuse.
We can "piggyback" on the ID checking done by the financial-services
industry, which has a lot more to lose than a domain-name vouching
service.
The many many millions of entities wanting a domain reside in almost
every country using different currencies and communication systems. A
consortium of postal services might be able to handle a centralized
registration process. Perhaps by correlating domain names with a
physical postal addresses. This would likely create a cottage
industry aimed at generating domain names used by abusers.
All of this applies only to those who need to "jump start" a new
legitimate ID. A good reputation can also be built by simply
sending lots of good mail and very little spam to many receivers
over a long time.
There remains an inherent conflict of interest, in addition to
compromised systems infiltrating otherwise reputable domains, and of
course more time is needed to accrue meaningful reputations. The TBR
mechanism affords perhaps 20 minutes needed to resolve reputations
from questionable sources.
Messages can be submitted as legitimate users, but likely originate
from compromised systems. Correlating source patterns and overall
volume determines the trend.
I think I understand what you are saying. Zombies are sending spam
not directly, but through their ISP's outgoing mail servers. The
ISP needs better authentication, rate-limits, etc. Most large ISPs
(aol.com, yahoo.com, etc.) have this type of abuse under control.
Some are better than others. Bad actors seek out networks imposing
fewer limitations and may see dial-ups or port 25 blocking as a
limitation, for example.
Blocking port 25 is not a complete solution. Compromised systems
can bypass such blocks. In the case of compromised systems, abuse
is widely diffused such that rate limiting and outbound content
filtering offers diminished relief.
I'm seeing very little spam from the authorized transmitters of
aol.com, yahoo.com, and most other legitimate senders. Eliminating
outgoing spam is not that difficult.
With an exponential trend, detection failures may be manageable now,
but are not likely to remain that way. Reliance upon content
filtering appears to be creating increasingly difficult to detect spam
and malware.
Agreed, but then this also runs into conflicts of interest. An
email-provider often does not want credit. TBR demands an
identity, a matching MailFrom, and an MX record. In addition, TBR
better identifies a problem source, and not just the last MTA
"holding the bag" so to speak.
An email provider that "does not want credit", or more accurately,
one that "is unwilling to assume responsibility" for what is sent in
its name, gets the reputation it deserves.
When the domains are large, blocking just at the domain will cause a
significant amount of collateral damage. When domain level blocking
is used to triage limited resources, head of queue blocking will
become increasing predominate problems.
The last MTA on the sending side (the "transmitter" in my
terminology) is more than a "bag holder". Of all the parties
involved in handling email, the operator of that transmitter is in
the best position to stop spam. Google.com cannot deny that its
transmitters are sending spam.
Google's problem is likely a conflict of interest. It wants new
accounts so much that it is making it too easy for spammers to sign
up for these accounts. The fix is not costly, however. They just
need to segregate the new accounts, and until they earn some trust,
use a different HELO name, and apply more rigorous rate-limits and
filtering.
A widespread abuse problem has been seen by most large email
providers, where even Google has improved their process. Eliminating
organizations who attract large numbers of new users seems counter
productive, as these domains also represent desired sources of email.
When spam must comply with grey-listing, transactions will
dramatically increase. The vast majority of email is spam where a
percentage can not be detected based upon content.
You may be right. I'm seeing a still small but growing percentage
getting past SpamAssassin.
I can see this heading to either of two endpoints. Either the
spammers will win, overwhelming the statistical filters and forcing
everyone to sign up with a large ESP with the resources to work out
individual arrangements with other large ESPs, or we find a way to
hold senders responsible, and as you say "shift the burden to the
transmitter", where it is 1000X easier to deal with.
If the spammers win, the anti-spam industry and a few large ESPs can
look forward to a long and prosperous future.
There is no profit allowing spammers to win. Spam represents a
growing waste of resources. Instead of one receiving server accessed
over a single path, email now often travels over two or more paths
through an array of servers needed to meet a resource demand. When a
small and growing percentage of spam slips past filters, and there is
a rapidly growing volume and diversity of sources, customers will
perceive a problem with the service.
I hope TBR succeeds, but I suspect that the cost to both senders and
receivers will outweigh the benefit of prompt delivery.
Deployment of TBR would be able to better ensure prompt delivery. The
cost to the transmitter needs to be increased, while also reducing
costs for the receiver.
The cost of publishing CSV records was very little, and it went
nowhere. Nothing wrong technically, just a lack of sender motivation.
I do not agree as CSV can offer path registration as well. This path
would be by name instead of a large IP address list. CSV can insure
an upper limit of a couple of transactions instead of 10 x N
transactions needed to construct possibly sizeable IP address lists.
CSV could have provided an authorization scheme using a sha1 base32
label published withn the authorizing domain. This approach scales to
any number of servers associated with a domain without increasing the
transactions performed by receivers. Too many wanted fairly dangerous
scripts risking DNS instead. : (
I think to get widespread adoption, the cost and risk must be not
much more than publishing an SPF record ending in ?all, and less
than the cost of publishing CSV records.
Ensuring messages are handled in a timely fashion will likely justify
rather extensive changes. As the problem grows, moving nearly the
entire burden to the transmitter will not be a difficult sell. It
will likely become perhaps the only choice.
I think senders may need a kick in the pants. I'm thinking of an
SMTP reject, with a message like "Sorry, your authentication records
are insufficient, and we cannot assess the reputation of <domain>.
Your message will be sent to our quarantine, but we cannot guarantee
delivery. If it is important, call your recipient to make sure they
got it."
There will not be any new notifications. No one wants to explain
anything to users. Its too damn expensive. Things will just become
increasingly broken.
-Doug
|
|