Re: Re: DNS lookup limit?

Alex van den Bogaerdt wrote:
> On Sun, Feb 27, 2005 at 08:05:31PM -0500, Radu Hociung wrote:
>
> > Alex van den Bogaerdt wrote:
> >
> > > > If X = 10 and Y = 111, it would mean all recipients check the first 10.
> > > Then just say: Everybody is required (not: expected) to have no more
> > > than 10.  Everybody is required (not: expected) to check all 10.
> > Precisely what I am trying to say. Thank you for putting it so clearly.
> >
> >
> > > > Between 10-111, it's up to the recipient if they are lenient, as the 
> > > > spec does not require them to be.
> > > This is the gray area I was talking about.
> > >
> > >
> > > Why be lenient at all?  
> > I think some leniency is needed because mistakes do happen, especially 
> > since we have some indirect mechanims (mx, include, redirect).
> Ack.  Be lenient in terms of 
>
> "geez, these guys must have made a mistake.  I'll let them know
> by rejecting the message with a specific error message and won't
> report them to the internet police just yet".
>
> but don't do
>
> "geez, these guys must have made a mistake.  Let's continue
> processing the record as if nothing bad happened unless it
> really gets out of hand".
Absolutely correct.

But please note that to find out if you should call the police or not, 
you'd have to do a few more lookups, to find out just how honest of a 
mistake it was ;) But this will be up to the local admins. If they have 
a process in place for reporting, they will do the monitoring. If not, 
they don't need to do anything further. I suspect that most sites will 
want to look into it, because when you get 1,000,000 emails with a cost 
of 10 queries each, you want to decide whether to (temporarily) 
blacklist, or continue to waste your resources (but this is a process of 
dealing with DoS attacks. Everyone should have a process)
I even think it's important to have a process like this, because when 
hackerX launches an attack, both victims are innocent, presumably. The 
recipient has to do the lookups, the SPF publisher has to respond to 
queries. DNS services cost money sometimes, and the cost is related to 
the number of bytes transfered. With SPF records the number of bytes is 
higher than with other DNS uses. (10mil queries * 100 bytes = 1GB). I 
know the queries can be cached, but when everyone on the internet is 
asking for your SPF record, there are still a lot of caches to fill, 
plus you may be using macros, so caching can be tricky.
But I do think it's a good idea that even if you see a match after the 
first 10 lookups (ie, it works out it wasn't an attack after all), you 
still return a PermError, so that all implementations behave the same, 
and you stay out of the grey.
> > Say that you have a vanity domain record that references 3 MX's (#1 has 
> > 2A records, #2 has 3A records, and #3 has 2A records). This adds up to 
> > 10 queries. If any of your service providers (work, home, cottage) adds 
> > an extra outgoing mail server, they probably won't inform you, but your 
> > record will break.
> That is correct.  I'm not sure about any implications of this
> (maybe the limit itself is a mistake).
The outgoing mail will be bounced by <> at helo:cottage.com to your 
vanity domain's MX. Hopefully cottage.com has their SPF record in order.
When one get the "Too many DNS lookups" error, perhaps they will do what 
 the wise do: google it first, to see if there is an easy solution.
That will bring them to the FAQ page, which may offer the following 
solutions:
1. Use your ISP's SPF record by including it, instead of guessing their
   MX architecture.

1a. Suggest to your ISP to optimize their SPF record (spfcompile) into
    a list of IPs, which will make it cheaper for both of you to operate
    your the mail systems.

2. Optimize your SPF record (spfcompile) with the -flatten option
   so that even queries not under your control are converted to IP
   lists. Be prepared to re-do this every time one of your ISP's
   changes their mail setup. This is probably very infrequent.

3. Consider routing all your outgoing mail through just one of the
   service providers (port 587 with SMTP AUTH)

4. Consider buying SMTP relay services from someone.

5. Consider running your own outgoing mail server(s).

Perhaps there are others, too. Early on in this thread I had a few more 
possible solutions:
> 1. implement a real-time DNS lookup table that can be accessed with the 
> exists:%{ir}.mailhosts.rr.com for instance.
>
> 2. add a single "A" record that resolves to a long list of IPs.


> Example: one MX lookup results in two domains, one of them having
> two A records.
>
> Without clear guidelines, interpretations of the spec could be:
>
> - count as one (one MX lookup is done)
> - count as two (MX records are a kind of indirection)
> - count as three (one MX lookup is done, need to do two A lookups)
> - count as one or three (depending on A records being included in
>           the additional section)
> - maybe others?
The limit is there to protect against waste of resources. Therefore I 
think it should be proportional to the cost of the resources. Ie, the 
total number of DNS queries, regardless of the type.
This is also very simple to formulate, so there will be little or no 
confusion.
Perhaps we should also have the reference implementation count the 
number of queries and output it. That will be taken as an authoritative 
count.
> Again, to eliminate any possible confusion, I think it is necessary
> to define _exactly_ what should be counted, how and why, for any
> possible scenario.  Perhaps this shouldn't be in the RFC, it could
> be in an implementation whitepaper.
Confusion should be eliminated. I agree.

But I think this limit needs to be in the RFC, otherwise compliance will 
be difficult to judge, and reliability difficult to guarantee.
> Almost all of the discussion on this and other forums stems from
> inprecise wording, uncertaincy or other kinds of confusion.  I'd
> really like to avoid any further uncertaincy as much as possible.
We're doing the clarification work right now :) I think.

Greetings,
Radu.