Frank Ellermann wrote:
Thanks for these interesting numbers. Wayne's 3*10 implicitly
gets a loop soon enough. For an overall limit like 40 that's
in theory dubious. OTOH the worst case of a loop would need
only 2 "real" queries, the remaining 38 answers come from the
cache, and that's okay for a PermError without any additional
"loop detection".
Don't be so quick with what a loop is, can be, an how long it can be.
Example:
a.com TXT v=spf1 redirect=b.com
b.com TXT v=spf1 include:c.com redirect a.com
c.com TXT v=spf1 include:d.com redirect a.com
d.com TXT v=spf1 include:e-with-timeouts.com redirect a.com
A simple loop may take 2 queries. But, when you hack together an
SPF policy with 40 queries, you have a better change to come up
with a longer, more convoluted loop. And the research also shows
that half of the most expensive records were really hacked
together, as they have syntax errors and redundancies.
One more benefit of a lower limit: these people will no longer be
tempted to put everything including their grandmother in their
SPF record, hoping that if they add enough stuff it will work
eventually.
Actually, I got the crazy idea to run all the domains seen in my
mailbox in the last 8 years. This is clearly more than I could
find in my maillog, which goes back some 35 days:
I found 42849 unique domain names there. Of them, 481 currently
publish SPF records without syntax errors. About 10 publish with
errors, another 20 no longer resolve. Very few of these are
faked, as I do not keep the spam, but only real conversations,
some of them coming from mailing lists, which are spam filtered.
One conclusion is becoming very clear:
The people involved with SPF that have the most expensive records
in the world. The rest of the world manages to keep their records
short and sweet.
This time the sample size went up to 481. 94.8% fewer than 10 queries,
5.2% more than 10.
1 108 22.45%
2 50 10.40%
3 43 8.94%
4 75 15.59%
5 61 12.68%
6 43 8.94%
7 30 6.24%
8 18 3.74%
9 18 3.74%
10 10 2.08% 94.80% <= 10 queries
11 6 1.25% 5.20% > 10 queries
12 7 1.46%
13 3 0.62%
14 2 0.42%
15 1 0.21%
18 2 0.42%
19 2 0.42%
23 1 0.21%
24 1 0.21%
Totals: 481 100.00%
The top offender list becomes very telling:
First, the looped domains: Many of them spammers of one denomination or
another.
bowyer.org | 02-101 | 00-33 | 00-33 | 01-34 | 00-00 |
fool.com | 03-101 | 00-24 | 00-25 | 01-26 | 01-25 |
cybrhost.net | 02-101 | 00-20 | 00-00 | 01-60 | 00-20 |
mxlogic.com | 03-101 | 01-50 | 01-50 | 00-00 | 00-00 |
clickaction.net | 02-101 | 00-50 | 00-25 | 01-25 | 00-00 |
wmsgaming.com | 02-101 | 00-50 | 01-50 | 00-00 | 00-00 |
foolcs.com | 03-101 | 00-25 | 00-25 | 01-25 | 01-25 |
foolsubs.com | 03-101 | 00-25 | 00-25 | 01-25 | 01-25 |
on-net.net | 01-101 | 00-08 | 00-08 | 00-75 | 00-09 |
tiscali.be | 03-101 | 00-20 | 01-20 | 01-60 | 00-00 |
aplatform.com | 02-101 | 00-12 | 00-13 | 01-50 | 00-25 |
nwfnews.com | 01-101 | 00-100 | 00-00 | 00-00 | 00-00 |
events.networkmagazi | 01-101 | 00-100 | 00-00 | 00-00 | 00-00 |
The non loooped domains:
mailzone.com | 04-24 | 01-02 | 00-00 | 01-18 | 01-03 |
I understand mailzone.com is just a test domain set up specifically for
the SPF test suite. Thus, it is skewing the balance towards a higher limit.
pobox.com | 03-23 | 00-01 | 00-00 | 01-18 | 01-03 |
Wow. The most expensive real record on earth. Or at least in my mailbox
(481 domains publish SPF, total domains queried: 42849). SPF involved.
w3.org | 02-19 | 00-00 | 00-01 | 01-14 | 00-03 |
sentex.net | 01-19 | 00-01 | 00-02 | 00-13 | 00-02 |
kitterman.com | 02-18 | 01-02 | 00-01 | 00-12 | 00-02 |
Scott Kitterman, involved with SPF.
netforum.com.br | 01-18 | 00-04 | 00-02 | 00-07 | 00-04 |
"Mike", active on dns-discuss.
cse.unsw.edu.au | 02-15 | 00-00 | 00-00 | 01-13 | 00-01 |
Neil Brown, active on dns-discuss.
AIG.com | 03-14 | 00-00 | 00-00 | 01-12 | 01-01 |
intl.paypal.com | 03-14 | 00-08 | 00-00 | 01-04 | 01-01 |
us.ibm.com | 03-13 | 00-00 | 00-00 | 01-11 | 01-01 |
paypal.com | 03-13 | 00-08 | 00-00 | 01-03 | 01-01 |
electrophobia.com | 05-13 | 01-03 | 01-04 | 01-03 | 01-02 |
Richard Parker, active on dns-discuss.
dcdi.net | 03-12 | 00-00 | 00-00 | 01-10 | 01-01 |
rypma.ca | 02-12 | 00-02 | 00-03 | 01-04 | 00-02 |
momentus.com.br | 02-12 | 01-01 | 00-00 | 00-06 | 00-04 |
Pedro Alves, not very active, but involved with the SPF list.
v2.listbox.com | 04-12 | 01-02 | 00-01 | 01-06 | 01-02 |
As involved with SPF as pobox.com.
microsoft.com | 03-12 | 00-01 | 00-00 | 01-08 | 01-02 |
reply.ebay.com | 03-12 | 00-08 | 00-00 | 01-02 | 01-01 |
ebay.com | 03-12 | 00-08 | 00-00 | 01-02 | 01-01 |
greatgulfhomes.com | 01-11 | 00-05 | 00-01 | 00-03 | 00-01 |
Terry Fielder, involved with SPF.
dwford.com | 01-11 | 00-01 | 00-00 | 00-07 | 00-02 |
Rick Cooper, involved with SPF.
astrum.ch | 03-11 | 00-01 | 00-02 | 01-05 | 01-02 |
net2phone.com | 03-11 | 00-00 | 00-00 | 01-09 | 01-01 |
dammfine.com | 03-11 | 01-01 | 00-01 | 01-07 | 00-01 |
Ben Damm, involved with SPF.
sourceforge.net | 03-11 | 00-00 | 00-00 | 01-09 | 01-01 |
mirapoint.com | 03-10 | 00-00 | 00-00 | 01-08 | 01-01 |
Marcia Alana Lovell, involved with SPF.
leave-it-to-grace.co | 02-10 | 00-00 | 00-00 | 01-06 | 00-03 |
Andy Bakun, involved with SPF.
nycap.rr.com | 02-10 | 01-01 | 00-00 | 00-07 | 00-01 |
returnpath.net | 10-10 | 00-00 | 08-08 | 01-01 | 00-00 |
vt.edu | 01-10 | 00-00 | 00-01 | 00-07 | 00-01 |
anarres.org | 03-10 | 00-00 | 00-00 | 01-07 | 01-02 |
Shevek,involved with SPF.
maine.rr.com | 02-10 | 01-01 | 00-00 | 00-07 | 00-01 |
match.com | 01-10 | 00-00 | 00-00 | 00-08 | 00-01 |
rhyme.com.au | 01-10 | 00-00 | 00-07 | 00-01 | 00-01 |
midsouth.rr.com | 02-10 | 01-01 | 00-00 | 00-07 | 00-01 |
If I remove those involved with SPF, I get 472 domains, 96.61% publish
records <= 10 queries. 3.39% are > 10 queries.
Perhaps SPF is not taken seriously because the promoters of SPF
encourage very muchly increased loads on the internet's DNS
infrastructure. Perhaps T-Online does not get involved with SPF
because they know that checking SPF records would put a strain on
their DNS infrastructure. Would you mind asking them, Frank ? You're a
customer and you vote with your money, so perhaps they will give you an
answer.
It would be very nice if those named above responded with:
1. Do you run your own SMTP server ?
2. Do you host your zone on your own DNS server ?
3. Do you run or administer your own MTA-based spam solution?
4. What is the nature of your experience in the world of SMTP and DNS?
Of course one can easily find if mail goes to some outsourced service,
and if the DNS queries go to some public DNS service.
Maybe a good way to make people aware of the load they put on other
peoples's DNS servers would be to start an "SPF hall of shame". I know
just enough about PHP and MySQL to make this happen. Anyone else willing
to help?
Radu.