For your perusal, the following accumulated stats for the past 6-7 months
that you might find interesting:
http://www.winserver.com/public/antispam/wcsmtpstats.htm
Only the past 3 months I had the logs turned on each day in anticipating of
getting some better analysis.
Engineering design notes, Log analysis and suggestions:
The rejection stats in the logs are based on what happens at each state
point in the classic SMTP protocol state machine:
http:://www.winserver.com/public/antispam/smtpstatemachine.jpg
Connect State:
The White/Black list and RBL checking is done at the connection level with
the rejection response delayed until the RCPT TO: state is reached.
This was done because we quickly learned that a certain group of spammers
(software) will continue to connect non-stop if the connection was
immediately and passively dropped. This information was being logged and
accumulated in a very high volume. In other words, it didn't seem like they
were 'learning" about the passive connection drops. By providing some SMTP
feedback, it helped resolves their "scheduler."
I selected the RCPT TO: state rather than earlier states to provide
rejection feedback so that we can log the target email address to analyze
any possible any false negatives issues by the RBLs.
Suggestions:
SMTP guidelines may suggest server implementations performing IP connection
filtering should provide a SMTP rejection response at the greeting,
HELO/EHLO or the RCPT TO: state points rather than passively dropping the
connection to help reduce continue attack by "dumb spammer software."
HELO/EHLO State:
In our original implementation, we provided no logic to check the client
host issued by the remote MTA. In 1999, we offered an optional strict
intranet client host domain checking for closed systems.
In August 21, 2003, we added new default logic to perform basic format
client domain format validation as defined by the RFC 2821 section 4.1.3
(Address Literal), this includes qualifying the usage of a bracketed IP
address provided by the client and checking the IP against for fraud IPs
(using your IP for example). As you can see beginning with the August
stats, this drastically helped reduce illegal entries at the HELO/EHLO
state.
The most common illegal usage is "HELO your.IP.address" without brackets.
Suggestion:
Obviously, the HELO/EHLO is probably the first of the most importantly state
point where technical compliance must be followed to help address the
illegal entries. There are few proposals on this subject of HELO/EHLO DNS
lookup validations.
I would like to suggest that the SMTP specs be changed to suggest that a
bracket IP address be used in all cases for all MTA clients.
This will allow for a quick non-DNS comparison against the IP connect
address. There is no need to provide a FQDN. However, it does, it must
match the connecting IP address.
Legacy clients will be handled by defined the SMTP server implementation.
My only suggestion here is that the SMTP puts it in writing that SMTP server
are NOT required to support old non-compliance software. Obviously, server
software will do a minimum here and allow the client to go next state, but
by adding the requirement to the specs, software vendors can begin to change
their client ware. Once possibility to offer an new positive response
code:
251 Warning: Illegal Client down. Future entry may be denied.
MAIL FROM: State:
Our new callerid verification method (WCSAP) was turned on November 23,
2003. This is a new hook into our SMTP server that is called once the MAIL
FROM: return path is provided.
The connection RBL rejection flag is passed to WCSAP so that it can log it
and skip the callerid verification process. In this case, a MAIL FROM: 552
response is provided.
As you can see in the November 2003 stats, before the 23rd, RBL provided the
strongest rejection method before the RCPT TO: state point is reached.
However, with WCSAP implemented, Caller ID verification now provides an
even higher rejection rate, nearly making RBL obsolete. This might suggest
that the RBL databases, I am using BL.SPAMCOP.NET, is doing an excellent
job verifiable with return path validation logic.
In our WCSAP implementation, it follows a standard SMTP sender process,
honoring MX logic, etc. It looks for the response code for a RCPT TO:
<original return path> and if a positive result is provided, it goes a step
further to check for open relay sites by issues an unique and random RCPT
TO: email address. Optionally, WCSAP can return a 552 for the open relay
site or allow it to go thru with a 250 response code.
The open relay checks works great! However, it is problematic with
YAHOO.COM which is inherently for all intent and purpose an OPEN RELAY site!
I have my RejectOpenRelaySite option enabled in WCSAP, so sorry YAHOO, fix
your system. Your system is NOT helping the internet email problem! I
put more weight on an open relay site being a spam site.
Suggestions:
By far, the MAIL FROM: is the probably the focus point of most anti-spam
proposals.
I view WCSAP as a "HOOK" to validate the MAIL FROM: state point. The
implementation is just one. If a better approach comes along that provides
the same but optimized "verification" results, fine! WCSAP might be
changed to use it.