[The following text is distributed under the GFDL licence and therefore can be added to any Web page about SPF. Maintained by Stephane Bortzmeyer but several contributors participated. Fixing the english mistakes is of course allowed.] You want to publish SPF records but you fear that some of your email will be rejected by SPF-checking sites? You have heard that SPF records not ending in -all are for sissies only but you're not sure? You have read some people's messages on spf-discuss saying there is no problem with forwarding and no problem with roaming users, but you do not trust them? You abide by the rule that significant changes in architecture should be field-tested before being rolled out? You are right. Legitimate mail can be lost if you deploy SPF carelessly (wether it will actually be lost or not depend on the policy of the SPF checker). Here is a way to test what the SPF checkers will see. You just need a DNS server able to log requests (we will use BIND 9). This is a patent-pending technique, invented by me. I intend to grant a RAND licence to anyone I like. The obvious prior art does not bother me, since the DNS records are not archived, it will be impossible to prove that other people used that technique before me. First, you need a *dedicated* name server. Unfortunately, there is no way in BIND 9 to log requests on a per-zone or per-view basis. You do not need a dedicated machine but you need a dedicated name daemon (you can use several IP addresses on one machine, if there is already a name server running, this is not covered in this HOWTO.) Then, you need to configure it to log requests. In named.conf: logging { channel "query_log" { file "/var/tmp/query.log" versions 3 size 2m; print-time yes; print-category yes; }; category "queries" { "query_log"; }; }; Read the BIND's ARM (Administrator Resource Manual) to get the details. Basically, It tells BIND to log every query to /var/tmp/query.log. You must *restart* the name server. Reloading is not sufficient. Then, you should have a /var/tmp/query.log file and requests sent to the name server will be logged in it. Then, configure your name server to serve the SPF tracking zone: options { ... (Other options) recursion no; }; zone "spf-tracker.example.net" { type master; file "spf-tracker.example.net"; }; And in the spf-tracker.example.net file, you put something like: $TTL 86400 @ IN SOA ns.spf-tracker.example.net. hostmaster.example.net. ( 2004112501 ; Serial 604800 ; Refresh 86400 ; Retry 2419200 ; Expire 86400 ) ; Negative Cache TTL @ IN NS ns @ IN MX 0 0.0.0.0 @ IN TXT "v=spf1 -all" ns IN A X.Y.Z.T And reload your name server. The last step is to actually delegate spf-tracker.example.net to ns.spf-tracker.example.net (not covered in this HOWTO). Then, it is time to modify your SPF record. Keep an eye on your favorite draft-of-the-day, I use draft-schlitt-spf-00 (http://www.midwestcs.com/spf/spf_classic_libspf2/draft-schlitt-spf-00.html) and read about macros. I have choosen the following string to make the actual DNS request: %{i}._.%{h}._.%{s}._.%{r}.spf-tracker.example.net although experience seems to indicate that %{r} is not widely implemented, probably for privacy reasons (it can gives too much detail about the SPF checker). To make the SPF checkers query that string, I add it in my SPF record, just *before* the final "?all" (since we are testing, we use a ? for the last mechanism): example.net. IN TXT "v=spf1 mx +exists:%{i}._.%{h}._.%{s}._.%{r}.spf-tracker.example.net ?all" That's all. Soon, you will see SPF checkers starting to query you. You can test right now by sending mail from various places and different setups to echo(_at_)generic-nic(_dot_)net or spfenabled(_at_)pobox(_dot_)com(_dot_) Here is a typical request, as logged by BIND: 25-Nov-2004 13:46:33.580 queries: client 208.58.1.193#17010: query: 80(_dot_)67(_dot_)170(_dot_)20(_dot_)_(_dot_)foo(_dot_)bar(_dot_)_(_dot_)bortzmeyer\(_at_)example(_dot_)net(_dot_)_(_dot_)r(_dot_)spf-tracker(_dot_)example(_dot_)net IN A - You can see that the SPF checker is 208.58.1.193 (a Pobox machine), its SMTP client was 80.67.170.20, which announced itself in HELO as foo.bar, the sender claimed to be bortzmeyer(_at_)example(_dot_)net(_dot_) Writing the Python or Ruby script to parse the log and make statistics is left as an exercice. In practice, it is more interesting to manually examine the log: each line is a problem, a sender which did not pass through your known SMTP senders. Either it was a worm spoofing your address (which means that SPF did its job) or a configuration bug that you should investigate before switching from "?all" to "-all". A few "Questions & Answers" about this HOWTO: 1) Q. Do you need your own DNS server? A. Yes. Specifically, the DNS server needs to be able to log queries sent to it, and you will eventually need to get access to that data to do your analysis. But any PC with a free Unix and an ordinary residential Internet access is sufficient (unless you want to test a SPF record for a very big ISP). 2) Q. Do you need a specific zone (sub-domain delegation) just for exists? A. No, but you would need a delegation if you want to use a different server than the one (ones) serving the parent domain. For example if you don't publish NS records for spf-tracker.example.net, the queries will go to the same DNS server as example.net. This may be fine for what you want but it has the potential to overload the main server. Reasons you might want a subdomain include: * Your domain gets a lot of queries (like a high-traffic website) and you only want to log queries related to the SPF exists: clause. Delegate the subdomain to another server and only turn on logging for that one. Remember that formatting the log for *every* DNS query is a costly task for BIND. * Your domain has many NS records pointing to many servers and you don't want to collect logs from many locations. Delegate the subdomain to only one server. If that server is down, the exists: clause will fail and processing should continue with the next term in your SPF record. That means you would lose logging/tracking info but it shouldn't alter mail delivery (unless you have actually populated the tracking zone with some data, something which we advise against later). 3) Q. Does the exists: clause have to refer to the same domain as the SPF record? A. No, you can write the exists: clause using any domain suffix you want. This is especially helpful if you are able to add TXT records but you can't add NS records to do delegations. Also this is a great idea for organizations that control hundreds or thousands of domains and want to track SPF results for all of them without creating hundreds or thousands of sub-domain delegations. The maintainer of the HOWTO even accept to host your SPF tracker if you agree and to send you back the results. Warning: this has huge privacy implications. To accept this offer means you really trust the person which manages the name server. 4) Q. Does the domain mentioned in exists: have to be populated with data? Will the domain ever exist, and will the exists: clause ever result in a "pass"? A. No, the zone you use for tracking doesn't need to be populated with any data (other than NS and SOA if you are using a sub-delegation as mentioned above). The example above is an empty tracking zone, and it was intended that way. It is expected that the exists: query will return "no such domain" (NXDOMAIN) result and won't pass. Where you put the exists: in your SPF record determines whether you will get all the queries for all SPF checks (if exists: appears first) or just the queries indicating the check is about to fail (if exists: comes right before ?all or -all, like we did in the above example). Some users might want to return some data to the exists: query. For example, you may want some local users to be exempt from SPF's sending restrictions, which can be done by supplying a wildcard entry that matches those queries. It's better (less confusing and error-prone) to do this as a separate exists: clause before the logging one, since then you can remove the logging clause later with no other changes. (Also, if you have delegated the logging subdomain to just one server, and that server goes down, this could change email delivery and cases that you want to pass via this method may fail. Any exists: entry that you might use to allow certain patterns through should be assigned to redundant name servers.) Really advanced SPF users may use similar techniques to set up a "stunt DNS server" that serves dynamic data instead of static data. For example, if you use a Perl program based on Net::DNS::Nameserver or a name server which allows you to add your own backend easily (like PowerDNS), you could set up a perl program that returns A records the first 10 times a valid return address (or client IP) is used, then switches to returning no results after the limit is reached for a certain return address (or IP). This would allow a roaming user to send several messages from a hotel but would stop spammers using fake local names and shut down spoofing of valid names after a short period of time. If you're going to the extra effort of tracking which local names or sending IPs have been seen before, you can also arrange for more organized logging or even email notifications to the administrator if you like.