ietf-mxcomp
[Top] [All Lists]

Re: RE: Can you ever reject mail based on RFC2821 MAIL FROM?

2004-04-24 06:56:58

----- Original Message ----- 
From: "Harry Katz" <hkatz(_at_)exchange(_dot_)microsoft(_dot_)com>
To: <ietf-mxcomp(_at_)imc(_dot_)org>
Sent: Saturday, April 24, 2004 3:05 AM
Subject: RE: RE: Can you ever reject mail based on RFC2821 MAIL FROM?


I asked for a scheme that permits correct rejection at MAIL FROM in the
face of non-SRS forwarders.  You've responded with a scheme that
requires universal SRS deployment.

Here are some interesting rejected transaction today:

legend:

state: where the hook was called
srvip: server ip
cip: client ip
cdn: client domain name (HELO/EHLO)
from: 2821 MAIL FROM
rcpt : 2821 RCPT TO:

Test performed and order:

sapfilter - internal while/black list
saprbl - rbl based lookup
sapspf - Sender Policy Framework
sapcep - Microsoft's Caller-Id Email Policy
sapcbv - Call Back Verifier

Here is the first time I saw a SPF and CEP working in concert!

20040424 07:43:31 -------------------------------------
20040424 07:43:31 version    : 1.61 / 1.54
20040424 07:43:31 state      : rcpt
20040424 07:43:31 srvdom     : winserver.com
20040424 07:43:31 srvip      : 208.247.131.9
20040424 07:43:31 cip        : 217.70.21.42
20040424 07:43:31 cdn        : hotmail.com
20040424 07:43:31 from       : <verify(_at_)testmail(_dot_)com>
20040424 07:43:31 rcpt       : <sales(_at_)winserver(_dot_)com>
20040424 07:43:31 testorder  : FLT RBL SPF CEP CBV
20040424 07:43:31 sapfilter  : pass (time:31)
20040424 07:43:31 saprbl     : testing 42.21.70.217.sbl.spamhaus.org
20040424 07:43:33 saprbl     : testing 42.21.70.217.list.dsbl.org
20040424 07:43:34 saprbl     : testing 42.21.70.217.bl.spamcop.net
20040424 07:43:37 saprbl     : pass (time:5469)
20040424 07:43:43 sapspf     : v=spf1 a mx ?all
20040424 07:43:43 sapspf     : neutral (time:6656)

Oh?  a neutral SPF result,  no trust, continue testing.

20040424 07:43:43 sapcep     : test from=testmail.com
20040424 07:43:45 sapcep     : test cdn=hotmail.com
20040424 07:43:46 sapcep     : <ep xmlns='http://ms.net/1'
testing='true'><out><m><indirect>list1._ep.hotmail.com</indirect><indirect>l
ist2._ep.hotmail.com</indirect><indirect>list3._ep.hotmail.com</indirect></m
</out></ep>
20040424 07:43:48 sapcep     : <ep xmlns='http://ms.net/1'
testing='true'><out><m><r>209.240.192.0/19</r><r>65.52.0.0/14</r><r>131.107.
0.0/16</r><r>157.54.0.0/15</r><r>157.56.0.0/14</r><r>157.60.0.0/16</r><r>167
.220.0.0/16</r><r>204.79.135.0/24
20040424 07:43:48 sapcep     : fail (time:4391)

oh look, a CEP test on the from and then CDN returns a reject!?

20040424 07:43:48 finaltest  : CEP GlobalResult=0 CodeResponse=554
20040424 07:43:48 result     : reject (0)
20040424 07:43:48 smtp code  : 554
20040424 07:43:48 reason     : Rejected by WCSAP CEP Fail
20040424 07:43:48 wcsap finish (16610 msecs)

Here is a quick 78 msec known spammer reject who really likes to issue local
domain spoofs!

20040424 07:45:26 -------------------------------------
20040424 07:45:26 version    : 1.61 / 1.54
20040424 07:45:26 state      : rcpt
20040424 07:45:26 srvdom     : winserver.com
20040424 07:45:26 srvip      : 208.247.131.9
20040424 07:45:26 cip        : 172.182.97.28
20040424 07:45:26 cdn        : localhorst.de
20040424 07:45:26 from       : <info(_at_)compuserve(_dot_)de>
20040424 07:45:26 rcpt       : <hector(_at_)winserver(_dot_)com>
20040424 07:45:26 testorder  : FLT RBL SPF CEP CBV
20040424 07:45:26 sapfilter  : line# 178: reject if localhorst in %CDN%
20040424 07:45:26 sapfilter  : reject (time:32) - blocked IP address
20040424 07:45:26 finaltest  : FLT GlobalResult=0 CodeResponse=550
20040424 07:45:26 result     : reject (0)
20040424 07:45:26 smtp code  : 550
20040424 07:45:26 reason     : Rejected by WCSAP Filter
20040424 07:45:26 wcsap finish (94 msecs)

I really like these:

20040424 03:09:35 -------------------------------------
20040424 03:09:35 version    : 1.61 / 1.54
20040424 03:09:35 state      : rcpt
20040424 03:09:35 srvdom     : winserver.com
20040424 03:09:35 srvip      : 208.247.131.9
20040424 03:09:35 cip        : 218.106.178.2
20040424 03:09:35 cdn        : smtpippkkfmkqk09o.privatemailserver.com
20040424 03:09:35 from       :
<epxhyuwhvikfmdikidshmmhm(_at_)p51120b0p6jx80t2(_dot_)sendmails(_dot_)com>
20040424 03:09:35 rcpt       : <andrea(_dot_)santos(_at_)santronics(_dot_)com>
20040424 03:09:35 testorder  : FLT RBL SPF CEP CBV
20040424 03:09:35 sapfilter  : pass (time:32)
20040424 03:09:35 saprbl     : testing 2.178.106.218.sbl.spamhaus.org
20040424 03:09:36 saprbl     : testing 2.178.106.218.list.dsbl.org
20040424 03:09:39 saprbl     : testing 2.178.106.218.bl.spamcop.net
20040424 03:09:41 saprbl     : pass (time:6609)
20040424 03:09:48 sapspf     : none (time:6406)
20040424 03:09:48 sapcep     : test from=p51120b0p6jx80t2.sendmails.com
20040424 03:09:54 sapcep     : test
cdn=smtpippkkfmkqk09o.privatemailserver.com
20040424 03:09:55 sapcep     : none (time:7703)

Start the CBV test!

20040424 03:10:02 sapcbv     : total mx records: 1
20040424 03:10:07 try mx     : sbbs.13trustee.com ip: 65.67.210.41
20040424 03:10:07 # connecting to 65.67.210.41
20040424 03:10:07 S: 220 sbbs.13trustee.com Microsoft ESMTP MAIL Service,
Version: 5.0.2195.6713 ready at  Sat, 24 Apr 2004 02:03:07 -0500
20040424 03:10:07 C: NOOP WCSAP v1.61 Wildcat! Sender Authentication
Protocol http://www.santronics.com
20040424 03:10:08 S: 250 2.0.0 OK
20040424 03:10:08 C: HELO mail.winserver.com
20040424 03:10:08 S: 250 sbbs.13trustee.com Hello [208.247.131.9]
20040424 03:10:08 C: MAIL FROM: <>
20040424 03:10:08 S: 250 2.1.0 <>....Sender OK
20040424 03:10:08 C: RCPT TO:
<epxhyuwhvikfmdikidshmmhm(_at_)p51120b0p6jx80t2(_dot_)sendmails(_dot_)com>
20040424 03:10:08 S: 550 5.7.1 Unable to relay for
epxhyuwhvikfmdikidshmmhm(_at_)p51120b0p6jx80t2(_dot_)sendmails(_dot_)com
20040424 03:10:08 C: QUIT
20040424 03:10:08 sapcbv     : 550
20040424 03:10:08 result     : reject (0)
20040424 03:10:08 smtp code  : 550
20040424 03:10:08 reason     : Rejected by WCSAP CBV
20040424 03:10:08 wcsap finish (33281 msecs)

Who says you can't validate a complete RETURN PATH address?

Here's another:

20040424 03:36:59 -------------------------------------
20040424 03:36:59 version    : 1.61 / 1.54
20040424 03:36:59 calltype   : SMTP
20040424 03:36:59 state      : rcpt
20040424 03:36:59 srvdom     : winserver.com
20040424 03:36:59 srvip      : 208.247.131.91
20040424 03:36:59 cip        : 221.139.186.250
20040424 03:36:59 cdn        : [221.139.186.250]
20040424 03:36:59 from       : <boknbcvbqsqd(_at_)flashmail(_dot_)com>
20040424 03:36:59 rcpt       : <junk(_at_)isdg(_dot_)net>
20040424 03:36:59 testorder  : FLT RBL SPF CEP CBV
20040424 03:36:59 sapfilter  : pass (time:32)
20040424 03:36:59 saprbl     : testing 250.186.139.221.sbl.spamhaus.org
20040424 03:37:02 saprbl     : testing 250.186.139.221.list.dsbl.org
20040424 03:37:04 saprbl     : testing 250.186.139.221.bl.spamcop.net
20040424 03:37:09 saprbl     : pass (time:9796)
20040424 03:37:14 sapspf     : none (time:5188)
20040424 03:37:14 sapcep     : test from=flashmail.com
20040424 03:37:16 sapcep     : test cdn=[221.139.186.250]
20040424 03:37:17 sapcep     : none (time:3000)
20040424 03:37:19 sapcbv     : total mx records: 1
20040424 03:37:19 try mx     : mail.flashmail.com ip: 216.239.161.152
20040424 03:37:19 # connecting to 216.239.161.152
20040424 03:37:22 S: 220 Welcome to FlashMail.com
20040424 03:37:22 C: NOOP WCSAP v1.61 Wildcat! Sender Authentication
Protocol http://www.santronics.com
20040424 03:37:22 S: 250 Command NOOP OK
20040424 03:37:22 C: HELO mail.winserver.com
20040424 03:37:22 S: 250 flashmail.com. Hello mail.winserver.com
(208.247.131.9)
20040424 03:37:22 C: MAIL FROM: <>
20040424 03:37:22 S: 250 Command MAIL OK
20040424 03:37:22 C: RCPT TO: <boknbcvbqsqd(_at_)flashmail(_dot_)com>
20040424 03:37:28 S: 550 No such user (boknbcvbqsqd) -ERR
boknbcvbqsqd(_at_)flashmail(_dot_)com not found
20040424 03:37:28 C: QUIT
20040424 03:37:28 sapcbv     : 550
20040424 03:37:28 result     : reject (0)
20040424 03:37:28 smtp code  : 550
20040424 03:37:28 reason     : Rejected by WCSAP CBV
20040424 03:37:28 wcsap finish (29000 msecs)

and if you notice, MOST of the 29 secs was tied up in the STUPID DNS
lookups!  9.7 for RBL, 5.1 for SPF and 3 seconds for CEP.

And just to be fair, here is CBV that returns a positive result:

20040424 00:04:49 -------------------------------------
20040424 00:04:49 version    : 1.61 / 1.54
20040424 00:04:49 state      : rcpt
20040424 00:04:49 srvdom     : winserver.com
20040424 00:04:49 srvip      : 208.247.131.9
20040424 00:04:49 cip        : 212.74.114.39
20040424 00:04:49 cdn        : mk-smarthost-3.mail.uk.tiscali.com
20040424 00:04:49 from       : <henry_obi3(_at_)handbag(_dot_)com>
20040424 00:04:49 rcpt       : <sales(_at_)santronics(_dot_)com>
20040424 00:04:49 testorder  : FLT RBL SPF CEP CBV
20040424 00:04:49 sapfilter  : pass (time:31)
20040424 00:04:49 saprbl     : testing 39.114.74.212.sbl.spamhaus.org
20040424 00:04:57 saprbl     : testing 39.114.74.212.list.dsbl.org
20040424 00:04:58 saprbl     : testing 39.114.74.212.bl.spamcop.net
20040424 00:04:59 saprbl     : pass (time:10688)
20040424 00:05:01 sapspf     : none (time:1828)
20040424 00:05:01 sapcep     : test from=handbag.com
20040424 00:05:03 sapcep     : test cdn=mk-smarthost-3.mail.uk.tiscali.com
20040424 00:05:04 sapcep     : none (time:3000)
20040424 00:05:06 sapcbv     : total mx records: 4
20040424 00:05:06 try mx     : mk-cpfront-6.mail.uk.tiscali.com ip:
212.74.114.8
20040424 00:05:06 # connecting to 212.74.114.8
20040424 00:05:06 S: 220 mk-cpfrontend.uk.tiscali.com ESMTP Service
(7.0.024.3-1) ready
20040424 00:05:06 C: NOOP WCSAP v1.61 Wildcat! Sender Authentication
Protocol http://www.santronics.com
20040424 00:05:06 S: 501 Syntax error on NOOP command
20040424 00:05:06 C: HELO mail.winserver.com
20040424 00:05:06 S: 250 mk-cpfrontend.uk.tiscali.com
20040424 00:05:06 C: MAIL FROM: <>
20040424 00:05:06 S: 250 MAIL FROM:<> OK
20040424 00:05:06 C: RCPT TO: <henry_obi3(_at_)handbag(_dot_)com>
20040424 00:05:06 S: 250 RCPT TO:<henry_obi3(_at_)handbag(_dot_)com> OK
20040424 00:05:06 C: RCPT TO: 
<wcsap-openrelay-test-123sxa23(_at_)alqwejad(_dot_)com>
20040424 00:05:07 S: 550 RCPT
TO:<wcsap-openrelay-test-123sxa23(_at_)alqwejad(_dot_)com> Relaying not allowed 
-
please use SMTP AUTH
20040424 00:05:07 C: QUIT
20040424 00:05:07 sapcbv     : 250
20040424 00:05:07 result     : accept (-1)
20040424 00:05:07 wcsap finish (17937 msecs)

Hey, nothing we can do about that.   But it was finally rejected at our DATA
hook:

**************************************************************************
Wildcat! SMTP Server v6.0.451.1
SMTP log started at Sat, 24 Apr 2004  00:04:48
Connection Time: 20040424 00:04:48  cid: 000058B3
SSL Enabled: NO
Client IP: 212.74.114.39 (unknown)
00:04:48 S: 220-winserver.com Wildcat! ESMTP Server v6.0.451.1 ready
00:04:48 S: 220-************** WARNING:  FOR AUTHORIZED USE ONLY!
**********************
00:04:48 S: 220-* THIS SYSTEM DO NOT AUTHORIZE THE USE OF ITS PROPRIETARY
COMPUTERS    *
00:04:48 S: 220-* AND COMPUTER NETWORKS TO ACCEPT, TRANSMIT, OR DISTRIBUTE
UNSOLICITED *
00:04:48 S: 220-* BULK E-MAIL SENT FROM THE INTERNET. THIS SYSTEM WILL
RESTRICT ACCESS *
00:04:48 S: 220-* TO CAN-SPAM (US S. 877) COMPLIANT CLIENTS ONLY.
*
00:04:48 S: 220
************************************************************************
00:04:48 C: EHLO mk-smarthost-3.mail.uk.tiscali.com
00:04:48 S: 250-winserver.com, Pleased to meet you.
00:04:48 S: 250-SIZE 5120000
00:04:48 S: 250-ETRN
00:04:48 S: 250-AUTH CRAM-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
00:04:48 S: 250-AUTH=LOGIN
00:04:48 S: 250 HELP
00:04:49 C: MAIL FROM:<henry_obi3(_at_)handbag(_dot_)com> SIZE=4893
00:04:49 S: 250 <henry_obi3(_at_)handbag(_dot_)com>... Sender validation 
pending.
Continue.
00:04:49 C: RCPT TO:<sales(_at_)santronics(_dot_)com>
00:05:07 ** WCX Process: wcsap  ret: -1
00:05:07 S: 250 <sales(_at_)santronics(_dot_)com>... Recipient ok
00:05:07 C: DATA
00:05:07 S: 354 Start mail input; end with <CRLF>.<CRLF>
00:05:07 ** WCX Process: SmtpFilterHookLoader  ret: 0
00:05:07 S: 554 Message Not Accepted by filter.
00:05:07 C: QUIT
00:05:07 S: 221 closing connection

I got a few more interesting ones, lets look at yesterday's look:

Here is a MAIL FROM SPF reject...

20040423 06:58:46 -------------------------------------
20040423 06:58:46 version    : 1.61 / 1.54
20040423 06:58:46 state      : rcpt
20040423 06:58:46 srvdom     : winserver.com
20040423 06:58:46 srvip      : 208.247.131.9
20040423 06:58:46 cip        : 69.105.129.52
20040423 06:58:46 cdn        : adsl-69-105-129-52.dsl.sndg02.pacbell.net
20040423 06:58:46 from       : <vegetate(_at_)email-o-mania(_dot_)com>
20040423 06:58:46 rcpt       : <asantos(_at_)santronics(_dot_)com>
20040423 06:58:46 ruid       : 20844
20040423 06:58:46 testorder  : FLT RBL SPF CEP CBV
20040423 06:58:46 sapfilter  : pass (time:32)
20040423 06:58:46 saprbl     : testing 52.129.105.69.sbl.spamhaus.org
20040423 06:58:47 saprbl     : testing 52.129.105.69.list.dsbl.org
20040423 06:58:49 saprbl     : testing 52.129.105.69.bl.spamcop.net
20040423 06:58:50 saprbl     : pass (time:4468)
20040423 06:58:54 sapspf     : v=spf1 ip4:216.65.64.1/24 ip4:216.65.3.1/24
a:send1.surgeweb.com mx -all
20040423 06:58:54 sapspf     : fail (time:4407)
20040423 06:58:54 finaltest  : SPF GlobalResult=0 CodeResponse=550
20040423 06:58:54 result     : reject (0)
20040423 06:58:54 smtp code  : 550
20040423 06:58:54 reason     : Rejected by WCSAP SPF Fail
20040423 06:58:54 wcsap finish (8969 msecs)

Now your assertion is "can we trust we reject?"  Is this a Forwarding
problem?

I say

1) Thats the domain's problem to resolve and,

2) if I don't get any complaints, I'm not going to worry about it.

In the mean time, we did ourselves a BIG favor and we probably did the
DOMAIN a big favor as well.

But the odds are good its not a forwarding problem because the domain is not
useing a Neutral or Softfail.

If it was, then possible the CBV will trap it.  Or maybe not

The point?

You CAN reject at 2821 to a very high degree.  When there are no complaints,
then there is something right about it, and even if the no rejects, then the
DATA stage comes into place.

All in all.

12% rejects at SMTP Helo due to syntax errors
40% suicidal drops at SMTP greeting due to lack of multi-line greeting
support by client

Of the removing 48% reaching MAIL FROM,  a delayed validation response is
issued until RCPT is known where at ~30% is rejected due to unknown RCPT or
no mail access for local user.

Once a RCPT TO is acceptable and before a response is issued, the remaining
34% (.48 x .70) are passed thru our wcSAP system where there is an average
51% is rejected with a method break down of:

FILTER - 11.5%
RBL - 75.6%
LMAP (SPF/CEB) 1.1%
CBV - 11.7%

(see http://www.winserver.com/sslinfo for complete stats)

Now why would I want to give this up and perform the same logic at RFC 2822
by first accepting all that mail?

How do we measure all this?

No complaints.   Thats really all we can measure it on.   By far, if there
is a problem, someone will report it "Hey, where is my mail?"   "Hey, why is
your system rejecting my mail?"  etc, etc.    Guess who is not complaining?
The spammers.

Anyway all this said,  I am still learning how to improve it.   You got
something better?  I'll be the first to implement it faster than I can bang
a dominoe chip!  But no way will the the push 2822 validation first concept
isn't going to do it for me and I certainly not going to put it into our
product in place of what we have.   I am not going to hurt my customer base
with this crazy idea and I'm certainly not going to contribute to this
unnatural process.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com










<Prev in Thread] Current Thread [Next in Thread>