spf-discuss
[Top] [All Lists]

DNS load research

2005-03-19 12:57:03
I did some research into my mail server's encounters with SPF to
gauge how many domains would be affected if we had set a maximum
limit on DNS lookups of 10 queries. Please find below the
fascinating results I found.

The procedure:

    I made a list of all domains I had contact with which publish
    SPF records. I did not include non-publishers, because
    there's no point. I ignored the SPF result that my server
    calculated (pass/fail/neutral/softfail). I removed 2 domains
    which resolved as PermErorr due to syntax errors

    These two domains were ignored:
    nerdshack.com v=spf1 ip4:209.189.235.44, ip4:209.189.235.43 ~all
    mailshack.com v=spf1 ip4:209.189.235.44, ip4:209.189.235.43 ~all


    I ran this list of domains through spfquery -loadsum to get
    summaries for the load presented by their respective SPF
    records. I sorted the output in order of decreasing maximum
    query load. This information is at the end of this message.

    I then used excel to create a histogram of the number of MAX
    queries per SPF record :

    Queries Number of domains   Percentage of total
    1              48                52.75%
    2              4                  4.40%
    3              14                15.38%
    4              6                  6.59%
    5              8                  8.79%
    6              3                  3.30%
    7              1                  1.10%
    8              1                  1.10%
    10             1                  1.10%
    11             1                  1.10%
    12             1                  1.10%
    14             1                  1.10%
    16             1                  1.10%
    101            1                  1.10%

    In total, I looked at 91 domains.

    It results that 94.51% of the examined domains publish
    records which require 10 queries or fewer to resolve
    completely.

    5.49% of the domains have SPF records more expensive than 10
    queries. I then looked at all these in detail to see why they
    need such expensive records and what could be done.

    I found that all 6 domains that published expensive records
    could reduce their records to between 1 and 6 queries.

    This detailed analysis follows:


jaydemail.com (currently limited to 101 queries by DOS measures)
    has a loop in its includes, I'll ignore it, with the note below.

    "v=spf1 mx ip4:66.28.139.0/24 ip4:66.98.160.93
    ip4:66.98.248.20 ip4:207.246.86.0/25 ip4:208.239.240.102
    ip4:208.239.240.73 include:newsletterinc.com
    include:squabblers.com include:newsletterplanet.com
    include:nichenewsletters.com -all"

    I browsed the domains, and they all resolve to the same
    website. The "About us" link reveals the following:

    "The iEntry Network consists of the best business-to-business
    web search engine, several targeted "niche engines" and email
    newsletters reaching over 6,000,000 unique opt-in
    subscribers. Total newsletter delivery is over 50 million
    emails per month."

    So this is a spammer using SPF to get through. I'm going to
    spend no more time here.


absolutemotion.com (currently 16 queries)

    "v=spf1 a mx ptr a:mail.absolutemotion.net include:cox.net,
    include:fastq.com -all"

    They have their own servers and also some ISP backup. cox.net
    does not publish SPF, should be removed. fastq.com publishes:

    "v=spf1 a:mailout.fastq.com a:sd.fastq.com
    a:hosting1.fastq.com a:hosting2.fastq.com
    a:hosting3.fastq.com mx:mailout.fastq.com mx:sd.fastq.com
    ip4:65.39.64.0/19 ip4:204.62.193.0/24 ip4:204.75.156.0/24
    ip4:204.238.176.0/24 -all"

    All servers under their (fastq.com) control, shuld be shrunk
    to list of IPs.

    This is a good example of goodwill, but insufficient time
    spent on creating an SPF record (including cox.net and the
    *comma* syntax problem prove this point).

    Due to the comma and the empty record, this domain will have
    to fix their record to get reliable mail flowing.

Verdict: can be reduced to *3 queries*

mail.classmates.com (curently 14 queries)

    "v=spf1 mx ptr include:classmates.com ~all"

    classmates.com: "v=spf1 mx ptr ~all"

    All servers under their control, shuld be shrunk to list of
    IPs.

Verdict: can be reduced to *2 queries*

v2.listbox.com (currently 12 queries)

    "v=spf1 redirect=listbox.com"

    There is no value to this redirect, it should be replaced
    with the same record as listbox.com:

    "v=spf1 mx a:dream.listbox.com a:emerald.pobox.com
    mx:fallback-relay.pobox.com ptr
    redirect=%{l1r+}._at_.%{o2}._spf.pobox.com"

    Clearly listbox trusts pobox, so the
    mx:fallback-relay.pobox.com should be replaced with
    include:pobox.com

    I've shown elsewhere that pobox.com's record can be shrunk to
    2 queries.

    This is an example of overzealous desire to showcase, and not
    enough design effort.

Verdict: can be reduced to *6 queries*

about.com

    "v=spf1 ip4:207.241.145.4 ip4:207.241.145.5 ip4:207.241.145.6
    ip4:207.241.149.197 ip4:207.241.148.39 mx a:mclist.about.com
    a:ablist.about.com a:jmail3.ops.about.com a:smtp.about.com
    a:mxc1s.about.com -all"


    [root(_at_)sun src]# host mclist.about.com
    mclist.about.com has address 207.241.145.5
    [root(_at_)sun src]# host ablist.about.com
    ablist.about.com has address 207.241.145.4
    [root(_at_)sun src]# host jmail3.ops.about.com
    jmail3.ops.about.com is an alias for www.about.akadns.net.
    www.about.akadns.net has address 207.241.148.80
    [root(_at_)sun src]# host smtp.about.com
    smtp.about.com is an alias for mail1.about.com.
    mail1.about.com has address 207.241.149.197
    [root(_at_)sun src]# host mxc1s.about.com
    mxc1s.about.com has address 207.241.148.39
    [root(_at_)sun src]# host -t mx about.com
    about.com mail is handled by 300 about.com.mail11.psmtp.com.
    about.com mail is handled by 400 about.com.mail12.psmtp.com.
    about.com mail is handled by 100 about.com.mail9.psmtp.com.
    about.com mail is handled by 200 about.com.mail10.psmtp.com.
    [root(_at_)sun src]# host about.com.mail9.psmtp.com
    about.com.mail9.psmtp.com has address 64.18.6.10
    [root(_at_)sun src]# host about.com.mail10.psmtp.com
    about.com.mail10.psmtp.com has address 64.18.6.10
    [root(_at_)sun src]# host about.com.mail11.psmtp.com
    about.com.mail11.psmtp.com has address 64.18.6.10
    [root(_at_)sun src]# host about.com.mail12.psmtp.com
    about.com.mail12.psmtp.com has address 64.18.6.10

    There is very much redundancy here. All the backup MX servers
    point to the same machine.

    Using CNAMEs in SPF policies does not work. So this record is
    broken, for all intents and purposes.

    This is another great example of goodwill but not enough time
    spent to design mail infrastructure. I don't think this
    domain would even notice if we reduced the DNS limit to 10
    and they started failing. they're bigger problems.

Verdict: can be reduced to *2 queries*


citizensbank.com

    "v=spf1 mx mx:12.46.106.20 mx:12.154.167.140
    mx:12.154.167.156 mx:12.46.106.21 a:mailgw02.citizensbank.com
    ~all"

    host -t mx citizensbank.com

    citizensbank.com mail is handled by 10
    zapprib00001002.citizensbank.com.

    citizensbank.com mail is handled by 10
    zappmab00001001.citizensbank.com.

    citizensbank.com mail is handled by 10
    zapprib00001001.citizensbank.com.

    All servers under their control, shuld be shrunk to list of
    IPs.


    This is another great example of goodwill but not enough time
    spent to design the SPF record. An MX mech with an IP address
    is an SPF syntax error, so this record is broken.

Verdict: can be reduced to *1 query*

Here is the summary of DNS loads presented by the domains in
question:

Domain               |Queries min-max|  TXT  | PTR   |   A   |  MX   |
----------------------------------------------------------------------
jaydemail.com        |     02-101    | 00-33 | 00-00 | 00-33 | 01-34 |
absolutemotion.com   |     02-16     | 00-02 | 00-01 | 01-09 | 00-03 |
mail.classmates.com  |     03-14     | 00-01 | 00-02 | 01-08 | 01-02 |
v2.listbox.com       |     04-12     | 01-02 | 00-01 | 01-06 | 01-02 |
about.com            |     01-11     | 00-00 | 00-00 | 00-09 | 00-01 |
citizensbank.com     |     03-10     | 00-00 | 00-00 | 01-04 | 01-05 |
lists.sourceforge.ne |     03-08     | 00-00 | 00-00 | 01-06 | 01-01 |
sadinoff.com         |     02-07     | 00-02 | 00-00 | 01-03 | 00-01 |
mminternet.com       |     03-06     | 00-00 | 00-01 | 01-03 | 01-01 |
pca.com              |     02-06     | 00-00 | 00-00 | 01-03 | 00-02 |
collegeclub.com      |     01-06     | 00-00 | 00-00 | 00-04 | 00-01 |
magomania.com        |     02-05     | 00-00 | 00-00 | 01-04 | 00-00 |
KEEPANDTESTTHEM.INFO |     02-05     | 00-00 | 00-01 | 01-02 | 00-01 |
forever-young.co.uk  |     01-05     | 00-00 | 00-00 | 00-03 | 00-01 |
WEWANTPRODUCTTESTERS |     02-05     | 00-00 | 00-01 | 01-02 | 00-01 |
hotmail.com          |     02-05     | 01-04 | 00-00 | 00-00 | 00-00 |
WANTEDPRODUCTTESTERS |     02-05     | 00-00 | 00-01 | 01-02 | 00-01 |
WENEEDPRODUCTTESTERS |     02-05     | 00-00 | 00-01 | 01-02 | 00-01 |
hotmail.com          |     02-05     | 01-04 | 00-00 | 00-00 | 00-00 |
bethkanter.org       |     02-04     | 00-00 | 00-00 | 01-02 | 00-01 |
whale-mail.com       |     01-04     | 00-00 | 00-00 | 00-02 | 00-01 |
transmission.vattenf |     03-04     | 00-00 | 00-00 | 01-02 | 01-01 |
whale-mail.com       |     01-04     | 00-00 | 00-00 | 00-02 | 00-01 |
telus.net            |     01-04     | 00-00 | 00-00 | 00-02 | 00-01 |
bluebottle.com       |     01-04     | 00-00 | 00-00 | 00-02 | 00-01 |
adelphia.net         |     03-03     | 00-00 | 00-00 | 01-01 | 01-01 |
salaf.every1.net     |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
pokemon-master.every |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
cani.every1.net      |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
humorenlared.every1. |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
acdsystems.bounce.ed |     02-03     | 00-00 | 00-00 | 01-02 | 00-00 |
eaglestar.every1.net |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
baldoria.ch          |     03-03     | 00-00 | 00-00 | 01-01 | 01-01 |
battleancients.every |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
lorisgiftshop.every1 |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
cerebralparese.ch    |     03-03     | 00-00 | 00-00 | 01-01 | 01-01 |
0-8-15.ch            |     03-03     | 00-00 | 00-00 | 01-01 | 01-01 |
freeideas.every1.net |     02-03     | 01-01 | 00-01 | 00-00 | 00-00 |
horafeliz.com        |     01-03     | 00-00 | 00-00 | 00-02 | 00-00 |
donator.se           |     02-02     | 00-00 | 01-01 | 00-00 | 00-00 |
asicware.com         |     02-02     | 00-00 | 00-00 | 01-01 | 00-00 |
fjbf.org             |     02-02     | 00-00 | 00-00 | 01-01 | 00-00 |
access-one.com       |     02-02     | 00-00 | 01-01 | 00-00 | 00-00 |
surfer.tc            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
2com.pl              |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
bkkmail.com          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
catchamail.com       |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
advantagefcu.org     |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
dcse.fee.vutbr.cz    |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
iqp.pw               |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
webtv.net            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
globe-rider.com      |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
check1check.com      |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
rehau.co.id          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
bromesfever.com      |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
emeraldynamics.com   |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
nieman.com           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
homeme.com           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
click.jcrew.com      |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
163.net              |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
dedeaux.com          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
abram.com            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
glick.com            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
openadoptioninsight. |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
mail.ru              |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
hoffmann.vg          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
autoworxs.com        |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
koredata.com         |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
dierickx.com         |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
unmail.com           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
sjodin.com           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
liphop.com           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
mjprimeproducts.com  |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
mutualanalytics.com  |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
parrott.com          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
nolimitsadvertising. |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
merrill.net          |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
mail2world.com       |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
pfadiwildegg.ch      |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
gmx.net              |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
vachon.net           |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
medeiros.com         |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
sprint.ca            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
tuten.com            |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
visionqwest-ent.com  |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
southern-cherokee.co |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
villanueva.com       |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
club-internet.fr     |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
alienplanet3.com     |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
bertomeu.com         |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |
mailcity.com         |     01-01     | 00-00 | 00-00 | 00-00 | 00-00 |


Regards,
Radu.


<Prev in Thread] Current Thread [Next in Thread>