ietf-mxcomp
[Top] [All Lists]

RE: Measuring MARID

2004-05-27 02:10:50

Damon:> No pre-caching (on the system) no parallelism.

I was using a 'for line in' loop shell script reading a text file doing
a dig and a grep on each $line and only looking up unique domains so as
not to throw the numbers off with my system caching the first lookup of
a domain.

 My DNS servers are near by... but I believe that is appropriate. 




Unless I've got the wrong end of the stick, 8877 digs in 125 seconds is an
average of 14ms, which must be a lot less than the average round trip time
to *their* nameservers, so these must be mainly cached records you're
getting. Sure, a proportion of a mailservers lookups will be locally
cached, but not nearly *all* of them, particularly for lower traffic
systems, unless TTLs are being ignored.

I had a look at some of my logs; for 340489 transactions (> 24 hours ago) I
got 12633 unique "sender" domains, of which 477 have SPF TXT records - a
lot of these have pretty low TTLs (about 70% are 3600) and I'd not done
this before (i.e. none cached), so it took about 2.5 hours, i.e. 0.75s per.
Which sounds about right to me.

I ran it again, and the script completed in 1.3 hours, or about 0.4s per
lookup, I expect if I ran it again right away I could get the time down
more.

Personally, I'm surprised that the proportion of domains we see publishing
SPF is as high as this (3.7%), but I agree with you about the number that
seem to be spam domains. I guess if you excluded another.com domains,
amazon and aol you'd have a pretty reliable spam signature :-)






<Prev in Thread] Current Thread [Next in Thread>