----- Original Message -----
From: "Alex van den Bogaerdt" <alex(_at_)ergens(_dot_)op(_dot_)het(_dot_)net>
To: <spf-discuss(_at_)v2(_dot_)listbox(_dot_)com>
Sent: Wednesday, March 10, 2004 8:16 PM
Subject: Re: [spf-discuss] Latest proposal re HELO checking: make HELO tests
optional
Are you by any chance doing dynamic RCPT validation _without_ making sure
the RHS is valid?
Yes, but all validation including the return path is suspended until the
RCPT is determined.
This is suggested in RFC 2821 in regards to any system implementation that
wishes to validate the return path.
I have some example SMTP trace sessions below.
Spammer connects to your server, claiming to be $random(_at_)$mydomain
You connect to -my-server- so _I_ have to waste my resources?
IMHO you need SPF... Once you've validated the RHS, you can do call
backs.
Not only are we using, SPF, we use about 4-5 other methods as well, put
together as a suite called "WCSAP".
First, we develop and market a commercial mail system (not just server). So
everything we do is optional. However, even though everything is optional,
we do design it to offer the best, optimized, most secured mode of
operation out of the box. The less support issues we get, more the better.
Since our product is predominately an Intranet, user authentication is the
heart of the system. In short, when it comes to SMTP, the basic rule of
thumb is:
- anonymous access can only send local mail (final destination)
- authenticated access have no restrictions (local or route)
That is the classic SMTP design and spirit of RFC 2821.
With our system, we offer many extensions:
For routing access,
- IP Relay Tables
- ESMTP authentication (user login)
- POP B4 SMTP
IP relay tables is old school stuff. Roaming users have problems with this,
so its only use, if used, for trusted routers.
ESMTP AUTH helped solve the roaming user problem.
POP B4 SMTP also old school stuff, it helped solve technical support issues
when users didn't have a clue on how to setup ESMTP AUTH or did not have a
MUA with ESMTP AUTH support.
ESMTP Options:
[X] Enable ESMTP AUTH
[_] Required for All Connections
[_] MAIL FROM must be local
[X] Ignore RBL rejects if authenticated
WCSAP options
(_) Validate at MAIL FROM
(o) Validate at RCPT
So in short, this whole business of addressing the spamming problem is
completely 100% for unknown anonymous access connections for local
mail/final destination transactions.
If the session is trusted using one the above methods, there is no need for
any additional validation.
if the session is not trusted, then the default option is to want to see if
the RCPT is for a local user.
Here is example sessions (prune miscellaneous)
**************************************************************************
Wildcat! SMTP Server v5.7.450.9b13
SMTP log started at Wed, 10 Mar 2004 12:43:47
Connection Time: 20040310 12:43:47 cid: 0002411A
SSL Enabled: NO
Client IP: 155.239.161.196 (unknown)
12:43:47 S: 220 winserver.com Wildcat! ESMTP Server v5.7.450.9b13 ready
12:43:56 C: EHLO santronics.com
12:43:56 S: 250-winserver.com, Pleased to meet you.
12:43:56 S: 250-SIZE 5120000
12:43:56 S: 250-ETRN
12:43:56 S: 250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
12:43:56 S: 250-AUTH=LOGIN
12:43:56 S: 250 HELP
12:44:03 C: MAIL FROM: <rgstier(_at_)worldonline(_dot_)co(_dot_)za>
12:44:03 S: 250 <rgstier(_at_)worldonline(_dot_)co(_dot_)za>... Sender
validation pending.
Continue.
12:44:08 C: RCPT TO: <sales(_at_)santronics(_dot_)com>
12:44:08 ** WCX Process: wcsap ret: 554 (Rejected by WCSAP Filter)
12:44:08 S: 552 Return Path not verifiable.
See the response to the MAIL FROM?
It says "Sender validation pending." It does this if the above WCSAP
option is set to validate at RCPT.
So it waits to see if RCPT TO: is even valid. In this case, it is so WCSAP
is called.
WCSAP is a pcode server-side scripting language (BASIC like) and it is a
concurent session, meaning it knows everything about the SMTP session. It
has all the parameters.
In this case, it came back in less than 1 second with a 554 response with a
"Filter" rejection reason.
That means it failed one of the ACCEPT/REJECT rules we have in first test
WCSAP does which is to check our internal filter file.
In this case, it found that the sender was spoofing the HELO domain with our
santronics.com domain.
The internal filter rules for this HELO spoof check is simple:
Reject if santronics.com in %CDN% and %CIP% != 208.247.131.9
Reject if winserver.com in %CDN% and %CIP% != 208.247.131.9
Reject if isdg.net in %CDN% and %CIP% != 208.247.131.9
Reject if catinthebox.net in %CDN% and %CIP% != 208.247.131.9
%CDN% is a macro for the client domain name (helo/ehlo).
%CIP% is a macro for the client ip address.
Now, we needed this because SPF had that HELO "hole" that I was talking
about. Now that SPF has a new provision to allow for additional HELO
checking, I can add the new logic to the wcSAP SPF implementation.
This will be the default otherwise "out of the box" the local domain spoof
checking in the filter will not be there. The sysop will need to define it
if they want to speed the local domain spoof checking.
What WCSAP does is (in this configurable order)
FILTER check
RBL check
SPF check
DMP check (turned off at the moment)
CEP check (Microsoft stuff)
CBV check (call back verifier)
I think it is important to note that when we first started our Anti-Spam
research, the first thing we did with wcSAP was just have the FILTER and
CBV:
Original WCSAP:
FILTER check
CBV check (call back verifier)
RBL was already at the SMTP level and it did the check first. So if RBL
failed at SMTP, it didn't bother to do anything else also. RBL was checked
at RCPT so that we can log the session parameters before rejection.
Also, WCSAP was called at MAIL FROM, as you would normally think it should
called from once the MAIL FROM was entered.
CBV proved to be very successful rejecting atleast 80-90% of all spoofing
spammers!
However, with over 25 beta testers, we quickly saw there were some major
overhead issues with the SMTP based call back.
So the next step was to add new logic to reduce the need to do the CBV.
That is when we added DMP, and within the last month, SPF and last week CEP.
Now, CBV is only called 2-5% of the total times of the remaining
transactions that pass all other test with no decisions.
Also, at SMTP, we do a minor syntax check for HELO domain literal IP
spoofing.
So if a system says:
HELO 208.247.131.9
HELO [208.247.131.9]
Both are rejected at SMTP for not having the brackets or attempting to use
our IP address.
About 15%of all transactions are rejected before of this believe or not.
On December 26, that HELO rejection skyrocketed to about 70-80% when we
added a multi-line Welcome greeting. I didn't show it above but it is
multi-line. At first, I thought these guys were actually reading the
warning about no spam, but its probably because their software is broken.
So by that time it gets to RCPT, only about 20% of all connections make it.
About 29% of the remaining arel RCPT are rejected for unknown users (who
real users who don't have email access or expired on our system).
We average about 2500 connections, 20% or 500 make it to RCPT where 29% are
rejected. So WCSAP is called for only 355 total transactions.
Now what is better?
1) Calling wcSAP for 500 possible sessions at MAIL FROM? or
2) Calling wcSAP for 355 possible sessions at RCPT TO?
Thats atleast a 29% reduction on LMAP, DNS, CBV, etc overhead.
I hope this explains it better. Its all the same - just optimized!
if you want to see our smtp anti-spam (wcsap) stats, check out
http://www.winserver.com/sslinfo
--
Hector Santos, Santronics Software, Inc.
http://www.santronics.com
We move RBL to WCSAP