That simple code for email verification as was discussed in thread
"RE: [Asrg] Several Observations and a solution that addresses them all"
will not work - it breaks mailing lists and is otherwise easily abused by
spammers. But I'v probably spent more time looking at various verification
and callback ideas then almost anybody else on the list, so I'v put my
notes in format that can be sent by email (see below). But beware these
solutions are also breakable by spammers and generally require modification
to SMTP prootocol (but same applies to most other proposals, I simply
have not seen any one idea that can actually work in the long run).
if you do not mind in a couple days I'll publish notes on callback email
retrieval as this has not been widely discussed yet. I have notes
for most of ideas/proposals presented so far to ASRG, some notes are in
good form (like below), some are not well organized (but I can try to find
time to do organize it as below and if people are interested, post this
to the list). I planned to post all this to website as well, never got
around to it so far...
From what is listed below, read #3. It does same as MAIL-FROM even more,
but does not break anything (i.e. mailing lists, forwarders).
1. Simple callback to verify email address
Technology/Idea:
When email is received and MAIL FROM command has been issues, the
receiving email server looks up MX for the domain entered in MAIL FROM
and then connects to that server to verify sender address exists
Origin: Postfix mailing list
Pros:
a. Easy to do using existing SMTP commands (like VRFY, EXPN or using RCPT)
b. Implementations already exist and works fairly well
c. Does not require modifications of SMTP or breaks anything
Cons:
a. Nothing is stopping spammer from using valid email address in FROM
even if they are not authorized to do it. Futhermore they could
still use different "From" header which will be seen by users as
the actual sender email address
b. Nothing is stopping spammer from having valid domain name and having
it answer yes to all verification requests (but valid domain name
makes it easier to track spammer if we know where his mailserver is
located permanently)
c. Email server operators do not want to allow email address verification
because this is abused by spammer to gather list of valid email addresses
Summary:
Usable now with no downsides but offers little protection against spammers
2. Callback verification with messageid tracking
Technology/Idea:
MTAs keep track of all outgoing messages for certain period of time
(say 5 days) and when email is received, the server looks up MX and
then tries to verify that email really came from that domain by check
with mail server through special verify command to see if messageid is
real. Variations require either messageid to be synchronized among all
MX servers or that verification be done for each MX servers listed or
that only first (MX 0) keep messageids and all other syncronize with it.
Origin: (not entirely certain, one of mailing lists, TBD)
Pros:
a. Unlike #1 it allows verification only if email was sent, so no chance
of email address harvesting
Cons:
a. Spammers can use valid email domains and verification - see 1.cons.b
a. Requires modification to SMTP (new command of extension to VRFY)
b. Requires mail server to keep track of messageids for all outgoing email,
either
with separate database or using log files
c. Breaks ability to send email directly and set from address unless email is
relayed through proper domain's from address
d. Syncronization of messageids among all MXs is difficult to accomplish and
all MXs being checked for messageid create too much delay
e. Message can not be rejected at mail transmission time as messageid is
not available then, so entire message HAS to be received and then go
through filter and if messageid is wrong its, then returned back.
f. This approach relies on that email messages are received within
certain period of time (no long delays during transmission), otherwise
the message tracking information may no longer be available. It should
be noted that because of this, all verification must be done by
receiving mail server and not by mail client, because mail clients
may check email several weeks too late and if this filter is installed
on mail client it could say that message is not authorized because it
checked email after message tracking information has already been deleted.
Summary:
Because of cons c & d this is not likely to be accepted by entire community
3. Complex callback verification requirying full message tracking server
functionality with dns extensions
Technology/Idea:
Either new protocol/system is used for tracking mail message or its
integrated into SMTP as new extension. Either way this is complete system
where outside autorized agent program can add and possibly delete message
tracking information into special database. A new Message Tracking
Header is introduced that has at least the following information:
- sender email, - unique message id, - name of message verification server
Additional extension to DNS is also necessary to list all possible mail
verification servers for the domain, this maybe either new field (MC)
or special agreed DNS name (mail-verify with CNAME or A) or with
possibly in-addr representation of agreed subdomain with A 127.0.0.1
When mail server has received an email and supports this functionality,
it should parse message tracking header. First it needs to verify that
listed sender email address is either one of mail transmission FROM or
the one in "From" header. It should then check on verification server by
by finding ip for listed name and then checking in dns for sender email
domain to see if its one of authorized verification servers. If it is,
then connection should be made to that server to verify that listed
message id and message from (and possibly date) are as then are in that
server, and if it is, tracking information should be deleted (special
command is issued from connecting party when its verified and n
mail can then be delivered.
When new message is being sent, it can be sent from arbitrary location
and arbitrary email address can be intered into "From" if necessary, but
email message must be entered into correct verification server for that
From domain and its up to the verification server to provide functionality
to verify that you're allowed to use that particular email account
- this can be based on ip (most simple way)
- if integrated into SMTP it it can be made part of AUTH
- Possibly certificates can be used (maybe with STARTTLS) with certificate
being issued and verified by user's ISP (so no central authority is needed)
Whenver possible verification server should not only check that user is
allowed
to send email with from address from that domain but that user is actually
who he says he is (i.e. his email address).
Possibly message tracking should include also details on how long message
ids are kept on message tracking server (expiration time).
Origin: Compilation of several ideas (spaml & other mailing lists) plus DNS
part taken from Vixies mailfrom. Actual proposal this full form is
mine.
Pros:
a. Provides functionality to verify that email was sent by particular
sender and he has right to use that email
b. System is similar to MAIL-FROM proposals but does not break mailing
lists or forwarders and in fact can be implemented with full backward
compatability
Cons:
a. Spammers can use valid email domains and verification - see 1.cons.b
b. Probably will require modification to SMTP protocol for verificaiton
c. New verfication server must keep track of all messages which requires
serious additional resources (cpu & storage)
d. Message can not be rejected at mail transmission time. See #2.Cons.e
e. Messages must be transmitted quickly (within period of time, tracking
is done). See #2.Cons.f for details
Summary:
Generally pretty good at determinning if user is allowed to send email
from mail domain (like RMX proposals) and does not break existing system.
It does require modification to all mail servers and to protocol to be
of any use. Since this does not provide global verification, spammer can
make new system in compliance with new protocol/extensions and self-verify
his emails, but having ip address of verification server can allow to
keep track of origin of emails sent by spammers.
Additional Notes (STSA):
Idea should be explored futher to use this with certificates, possibly
approach is to issue certificates to verification servers and they can
issue certificates to users.
Possible if emails are groups together on the verfication server by who
sent it, a special command can be established to check how many emails
this particular user has sent to determine if emails are sent in bulk
(but because sender is providing his own verification, he can easily lie...)
It would usefull to be able to deleted message tracking when message
has been received (i.e. destination mail server can delete it after
verifying the message), unfortunetly this does not work well with mailing
lists where multiple receipients receive the same message and all need
to verify it. But possibly sender can specify if email is certainly for
one user or possibly after delete command is received, message would not
be deleted immediatly but faster then usual.
3b. Complex callback verification with dynamic dns used for verification
Technology/Idea:
This is not idea on its own but variation of #3, so read #3 first.
Particular to this variation is the idea that dynamic dns can be used
for verificaiton which would not require modification to SMTP. In particular
the setup would be such that when some server/user sends out email, it contacts
special one of specially listed dynamic dns servers (as listed with special
record for a zone its using for FROM) and adds new zone to it being actual
messageid. The zone would include, several records - EMAIL (TXT record with
actual email address), VRFY (either A or CNAME of verification server).
SOA of the zone would already include date and expiration info, which is
usefull. DNSSec can be used for secure verification and to establish identity.
Origin:
My unpublished idea to go along with complex verification.
Late addition - around March 02, 2003 (most other notes are from Nov/Dec
2002)
Pros:
a. Everything listed for #3.Pros applies here
b. Particular to this is that this would not require modification to SMTP
protocol
c. Since DynDNS software is already written and widely used, this can
be implimented fairlyquickly
Cons:
a. Everything listed for #3.Cons applies here
b. This will most likely require dedicated ip address for dyndn sserver
as such this can be a problem for those who currently run everything
(dns, mail, etc) from one ip address
c. I have serious doubts that DynDNS can handle such a high frequency
of updates/additions that would happen (though retrievals would be
no problem) and that it can actually handle millions of subdomain
zone files. STSA and preferably ask dns experts
d. Even without modification to SMTP its still necessary to modify
transmission
servers to enter correct info about eash message as well as modify
receiving servers to check for special header, etc. So in the end
this still requires changes to how SMTP works
Summary:
While pretty neet idea, it probably will not work because of con c
3b. Complex callback verification with modifications to MAIL FROM
Technology/Idea:
This is something to go with #3, and is not an idea on its own. Read #3
first.
Main addition here is that MAIL FROM as used during transmission can be
modified
to indicate that message has tracking information. This can be done as
extension
(ESMTP extension to MAIL FROM). Or it can also be done by modifying actual
email
address, so instead of user(_at_)domain(_dot_)com, this becomes:
verification(_dot_)server(_dot_)ip(_dot_)address+messageid++user(_at_)domain(_dot_)com
Hopefully everything after the + can be ignored. But because full FROM address
is passed along all between all servers used in relaying the message, it'll
end
up going up to the very end (receiving server) even if servers in between
do not support new extensions. The receiving server that does support the
extensions can use this information to bounce messages during transmission
instead after email is already received and email headers can be retrieved.
Origin:
My idea to go along with complex verification, part of original notes
separated
because this particular extension can break mailing lists
Pros:
a. See #3.Pros
b. Additionally bad email can be bounced during transmission in most cases
c. This basicly automaticly cloaks sender's email address which makes it
more difficult for email harvesters.
Cons:
a. See #3.Cons
b. This would fail with many mailing lists and there everything would
still rely on special header. This can not be used as substititure
for this "tracking" header
c. Very long MAIL FROM can be a problem with old email servers.
Summary:
Because of cons b & c, probably better to go with ESMTP extension but
still STSA
----
William Leibzon
Elan Communications Inc
william(_at_)elan(_dot_)net
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg