Re: Re: DNS load research

Andy Bakun wrote:
> Would anyone be complaining "additional email transactions are bad for
> the DNS! No one will want to send greater amounts of email once they
> find out the load it puts on their DNS servers"?  What if MX queries for
> SPF checking are considered exactly as expensive as MX queries for
> sending email?  This is why I weighted mx low initially, because if MX
> isn't cheap, then sending email isn't cheap to begin with.  But I could
> see it going significantly higher.
Spam traffic is growing much faster than real email traffic. The real 
email traffic grows roughly proportionally with the number of new 
subscribers to the Internet, while (attempted) spam grows proportionally 
with the effectiveness of SPAM filters. As long as the profitability of 
spam remains even marginal, the levels of overhead handled by the MTAs 
and DNS will grow (much faster than the levels of email between friends, 
 partners, mailing lists).
> My weights were not solely based on some raw "number of queries" count,
> but on how that mechanism fits into the whole system, just not DNS
> (otherwise, the weights are nothing more than a query count, and we
> should just count queries -- I'm trying to think about things
> differently here).
I like the idea of weights, but it it is a purely academic exercise, 
because at run-time it is difficult or impossible to calculate the real 
expensiveness of a record. The checker can try estimating it, but it 
will probably not be nearly accurate enough to be useful. This is 
because of DNS caching.
In a large installation, the DNS cache and the incoming MTAs may be 
different physical machines. The traffic between them is not very 
relevant because it's on the ISP's network, so it doesn't cost much 
money, once they have a gigabit ethernet installed.
However, cache requests that result in traffic on the ISP's connectivity 
to the backbone do cost money.
It is therefore difficult if not impossible for the checking code 
running on the MTA machines to estimate if the query it is about to do 
too result in a packet sent to the backbone of if it will be served from 
the cache.
The average cost of the backboe traffic is proportional to the 
complexity we allow SPF to have. It can be estimated using weights for 
the different queries, but this is a design-time estimate, as at 
run-time it cannot be done reliably.
I still think the macros can cause far more backbone traffic than the 
mechanisms themselves, because macros are much less likely to be cacheable.
> I don't think anyone disagrees that individual senders need to determine
> for themselves (perhaps, if you're an ISP who supports vanity domains,
> with input from your customers who might use include) if they need to
> make the robustness/load tradeoff in their records.

Vanity domain owners, who are two or three steps removed from the 
worries of running a DNS+MTA combo, will have no incentive to make their 
record cheaper.
An SPF publisher can only lighten his DNS load a _little_ by making his 
DNS simpler, because the bulk of his DNS load is still queries he has to 
do due to other people's SPF records.
Another thing that really bothers me is the potential for malicious 
'punishment':
Say that I really don't like domain X. My own email is forged like 
there's no tomorrow, so lots of MTA out there execute my SPF record.
If I make my record "+ip4:my_ip -a:a.X -a:b.X -a:c.X ... and so on to 
the allowed limit, my record will cost X dearly. I will never be 
blacklisted for doing this, because my denying X to send on my behalf 
appears legitimate. also, I'm not requesting a number of queries within 
the limit, so I'm not DOS's any recieving MTA.
What's even worse, is that poor domain X will have to answer all those 
millions of queries and not even know why he's queried so heavily.
replace X with microsoft.com, and you will see that a LOT of people 
would like to implement this strategy. But poor little ohmi.org will be 
put out of business very quickly if this happened to it.
What's even worse than that is that I can do something like "-a:%{i}.X" 
and "-a:%{i}.%{l}.X" (IP and username), and ensure that the queries are 
not cacheable, so the victim is hit much, much harder.
You'll no doubt say that X will then implement some kind of blacklisting 
on their DNS server, and refuse to respond to queries from some IPs.
Very well, so now all those MTA's trying to resolve my record will:

A. time out 10 times while waiting for a response from X that will never 
come.
B. will never be able to find out any NS info from X. So they can't find 
out what X's web server is, what it's MX record is, and so on. It will 
look to the victim like X does not exist any more.
I want to be very very careful while standardizing something this dangerous.

Greetings,
Radu.