ietf-asrg
[Top] [All Lists]

Re: [Asrg] Quarantines and block lists

2007-01-28 01:00:17

On Jan 27, 2007, at 6:43 PM, gep2(_at_)terabites(_dot_)com wrote:

Here is my proposal. I'm sure I haven't thought of everything so please tell me where it breaks or how spammers are going to game it. The goal is to stop the majority of the spam while not loosing ANY legitimate email (broken MTAs don't count).

To begin with, crude IP blocks will ALWAYS lose legitimate mail, because a single IP address can send both legitimate and zombie- generated mail. I propose that NO ISP-based software will be capable of differentiating one from the other... but the recipient possibly will know how to do that, at least for familiar senders.

I guess I didn't think to include a definition for "Lost Email".

Lost Email is any email that is sent but not presented to the user and does not generate a DSN to the sender. When referring to a receiving system loosing email it is assumed that the sending system is following standard procedures.

This is not intended to replace existing blocklists but to handle the initial onslaught from a spam source until the source can be properly categorized and listed.
Proposal #1 is a quarantine for new sources:
Connections from unknown source addresses would be initially rejected by returning a temporary error code (4xx) for a probation period. The probation period could be as short as 5 minutes depending on how fast the blocklists can respond. The reject would occur after the recipient list is accepted so the recipients can be checked against local spamtraps and valid user lists. After the probation period, global blocklists would be queried to find the current reputation of the source.

While spamtraps and valid user lists are helpful sometimes, they will not trigger if the spammer is using "fresh" E-mail address lists which are also devoid of spamtrap E-mail addresses.

Is there evidence that spammers have 100% spamtrap free lists? Where are they getting their addresses from and why can't we seed spamtrap addresses there? It cost almost nothing to try different forms of spamtrap seeding. I'd like to open this part up to all the armchair anti-spammers with a distributed process. The trap submissions could be scored on the basis of their verified spam counts and derated for getting non-spam.

Proposals #2 & #3 are for a fast global block advisory and distribution network Any email addressed to a spamtrap would generate a report to a local trap handler. The local trap handler would forward appropriate reports to the regional or global blocklist managers. The blocklist managers would publish advisories based on the reputations and counts of the traps that were hit by the sending IP.

IP address blocking, again, is simply too crude. A further and more serious problem is that you have basically no way to be 100% sure of who the "sending IP" actually is.

For just one example, Yahoogroups

That's obviously a non-starter. Yahoogroups is a subscription based service. A valid spamtrap would not be subscribed to any group (unless it was designed to catch spammers harvisting group addresses in which case it would take special precautions to exclude group related emails from the spamtrap reports).

Or let's say that Aunt Gertrude's machine gets infected by a worm, which starts sending out E-mails using Aunt Gertrude's account at the ISP, via the ISP's mail server. Are you going to block everything coming from that mail server? What if it is a VERY large ISP? Obviously, it would have been nice if that ISP's outgoing mail rules had caught the messages, but in this case let's recognize that they didn't.

Spam has to be stopped somewhere. But read the full set of proposals... If a site has never received email from that IP before it would quarantine the first emails from that IP as in proposal #1. The automatic spamtrap network would generate an advisory that the IP is sending UBE. Then depending on the reputation of that IP from all sources the site would choose to a: continue the quarantine, b: accept the email and possibly apply additional filtering or c: reject the email from that IP.

If the source is emitting a sufficient volume of spam this happens before the initial probation period expires.

Again, that's part of what I said about spammers engineering their mails so they squeak under whatever threshold is established to trigger spam alarms. That can be done on content, sending rate, or any other generally/widely used criteria.

So I've forced the spammer to send less spam. This I call progress.

DNS is probably a poor choice for distribution of the global fast response block advisories.

Obviously, since a change in status doesn't propagate quickly enough.

I think a form of flood distribution through a mesh of peer nodes would be more robust. The distribution would feed local databases that could be queried directly or through a DNS front end. The mesh can handle data from multiple sources and needs no central authority.

And how do you propose to prevent worms from targeting and flooding or doing other Denial of Service attacks on that mesh?

What port and IP address is the worm going to target? You won't know unless you've talked with the admin of that node and even then you might have a private port that only you know about. And even if you do manage to take out one node (or a dozen) the mesh survives and routes around the damage. The affected nodes re-negotiate the links with their peers and are back on line in short order.

Proposal #4 is back reporting unsolicited email:
If email addressed to certain spamtraps is received from a sender that has registered to accept back reports,

Original sender? A forwarding sender? An aggregating sender? How does the objecting MTA make sure?

That should be "sending MTA". Although an ISP might register an alternate protocol to report suspect email for an entire net block. The registration is primarily to prevent sending reports directly to a spambot.

accept the mail but return a specific code in the response that the sending MTA can interpret to make a determination if the sender is spamming. Alternatively, the site may register a different protocol to use for the back report. The spamtraps used for back reporting should be different from those reported to global blocklists to prevent disclosing the traps to rogue ISPs that would forward them to spammers. Only some of the traps would be used for back reporting to any given network or host.

I'm not convinced that returning ANY "spamming!" status to any imagined source is really worthwhile. To begin with, I think that most such reports will be discarded or archived, and just create additional worthless traffic on the Net. Secondly, if the "back reporting" can be misdirected (by counterfeiting E-mail or ISP addresses) then the creative spammer can induce the bounce message MTA to effectively be the "proxy" "sending source" of the spam, tricking it into sending the bounce message to the third party. (I used to get a fair amount of such apparently-intentional bogus bounces... and indeed, I still get a lot of antispam bounces into my "catch-all" E-mail account at a vanity domain, generally resulting from spammers who are counterfeiting randomly generated "From" addresses supposedly within my domain).

1. Read in this proposal where I said "registered". 2. Do you know what a response code in SMTP is? 3. I never "bounce" anything.

Proposal #5 is a quarantine while a site is listed in a block advisory. Established sources that get listed in local or global block advisories should have their mail quarantined while the mail is sorted out. The best place to quarantine the mail is on the sending server. It's already queued there and if it is eventually determined that it should be rejected,

And on what basis or set of rules do you propose that such determination be made? I will suggest that (at least for some non- obvious cases) that determination could be difficult or even impossible to do accurately.

The determination to quarantine email while waiting for additional information to become available or the possibility that the sending site will clean up the spam first should be easy to make. Are you perhaps referring to the rules for implementing proposal #2?

Apart from (as previously noted) the problem of determining after the fact with any real accuracy just who the "sending server" actually is, particularly for multi-hop mail deliveries. It's easy enough to forget that not all E-mail deliveries are simple user->ISP->ISP->user transfers.

First you have the intranet delivery on the sending side

        user-> (users ISP internal servers) -> Sending MTA

Then you have the internet delivery

        Sending MTA ----> Receiving MTA
or
        Sending MTA ----> (open relay) ---> Receiving MTA

And finally the intranet delivery on the receiving side

        Receiving MTA -> (receiving ISP internal servers) -> destination

I am only concerned with the internet delivery. Anything else the ISPs can handle on their own. So all internet email deliveries are simple ISP->ISP transactions.

Vanity domains that provide mail forwarding are just one example of a way that intermediaries enter into a mail delivery chain. Aggregators/forwarders like Yahoogroups are another.

If the vanity domain allows you to create hundreds or thousands of forwarders to addresses you don't own to make it look like a spammer site it deserves to be blocked.

that can be done with a 5xx on the next delivery attempt instead of sending a bounce.

And what if the next E-mail coming from that IP address is legitimate, non-spam E-mail?

If the decision is to reject all mail coming from that IP then returning the 5xx response will cause the sending MTA to generate a DSN that will be returned to the legitimate sender.

If the sending ISP is on their toes an offending user or host will be shut down quickly and the remaining mail will be delivered when the block advisory is lifted or expires.

And what happens to their legitimate outgoing mail in the meantime?

It sits in the quarantine until the block advisory is lifted and then gets delivered normally. Nothing is lost except for some sleep by the admin on duty for the sending ISP.

Local whitelists can be used to allow mail to  bypass the quarantine.

That's fine, but (again) I believe that traditional, simple "whitelists" are not nearly adequate. I believe that a viable whitelist MUST comprise a recipient/sender pair of addresses, AND additional criteria regarding what the recipient expects to receive from the sender in question.

More to the point, once THAT is done, I believe that IP address blocking becomes essentially superfluous (and indeed, COUNTER productive).

None of my proposals say "block" anything. I say issue advisories in proposal #2, some users might want that information. I propose quarantining email when it's looking suspicious. The sending ISP knows where the email came from which is something that the receiver cannot know in all cases. If anybody is going to decide that email is spam it is the sending ISP that has the best information to make that decision. Proposal #4 provides the one key piece that the sending ISP doesn't have to determine that their user is sending UBE (aka spam).

If you as an ISP are offered these free tools to help control spam emanating from your network are you going to say "no thanks, we'll just send the spam and let everybody else clean it up"?

-- Dan Oetting



_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg