Hello everyone,
I'm going out on the limb here. So everyone with a heart, please don't
jump on me, I'm just providing my technical opinion here. This is a little
long winded, but its done with a purpose, hopefully with goal of reaching
people (especially developers of mail software) who feel the same way I feel
to finally address the spammer problem once and for all. So brew of pot of
your favorite coffee, a smoke or whatever you do that relaxes you.
I recently learned of this group via a cyber space article who were talking
about what efforts are being done to combat the growing spam problem. The
article was talking mostly about the basic approach of various method filter
methods, but it concluded with a statement that the solution may also
include a fundamental change to the SMTP system being orchestrated by the
new AETG.
"Finally! YEEPEE!" I said! Finally, we are getting to the heart of the
problem instead of all the kludgy filtering solutions that a) didn't work,
and b) to a large extent played into the hands of spammers. I wanted to be
part of this process that will fundamentally finally change the outdated
SMTP system!
So I wrote Yakov and others with eagered baited breath telling them I want
to HELP! We have a pretty big commercial system but I didn't really tout my
own horn. I wanted to really be part of the process to help define the next
generation software. So I joined this group with one and only one goal in
mind, to help change the outdated SMTP specification in such a way that
would immediately stop the spamming problem.
Unfortunately, what I found here was a little disappointing. I was little
lost with whats going on with this working group. Instead what I saw here
is a slew of proposals which to me, basically beats a dead horse and will
not solve the problem.
It makes no sense to me to study or analysis the "patterns of spammers" or
what techniques they use, etc, or what "tricks" they have up their sleeves,
or hearing what spammers had to say in this group. All I saw here weree
efforts that attempt to "play games" spammers using filters or other items
that is basically playing into their hands. Sure, we can learn about them,
but does it solve the problem? I don't think so.
In the end, working solutions will be based on what the software developers
and vendors see as clear, sound technical design solutions and more
importantly, they will only allocate resources that addresses and solve the
main problem head on. They are not going to implement dozens of these
proposals. They will see the best and work with that.
To me, we know what the problem is - the SMTP specification! We change it
and the problem is essentially solved!
Sure, absolutely! I understand the "spirit" of the internet of keeping a
open standard and the resistance to deviate from the current standards.
However, it is this "spirit" that we need to let go and admit where the
problem of spams really lies and ultimately, the solution will need to be
finally be based on the fundamentals of the solid mail distribution system.
I say this because of past experiences with standard mail systems that
resisted change. If things don't change, further chaos will evolved and
ultimately, may threathen the very existence of the SMTP protocol, including
any say this IETF working group may have.
So allow me to share with you the story of Fidonet! - its raise and fail and
what we can learn from it.
The Raise and Fall of Fidonet.
Lurking this working group reminds me of the old pre-internet Fidonet days
where Fidonet developers and Fidonet standards administrators were two
conflicting breeds with two different mind sets.
Although we didn't have the major problem of "spamming" like we have today
in the internet mail market, Fidonet technological progress was hampered
with administrators (usually non-developers) conflicting with developers
eager for progress and the implementing of new and better methods that both
addressed and solved the issues of its time.
Understandably, administrators were more concern about standards and
compliance with legacy systems while developers were concern about changing
outdated standards compliance issues to begin offering and supporting new
methods that were beginning to threathen Fidonet existence. This conflict
help kill Fidonet as developers and system operators moved on to newer
things - such as the internet.
I am 100% confidence SMTP as we know it today can suffer some of the same
consequences. In fact, we are already to beginning to see this with the
larger ISPs beginning to ignore RFC SMTP guidelines and basically doing
their own thing.
So hear me out. If I am off base here, I will pack up and go away because
I don't want to waste time with kludging an outdated system and play games
with spammers. I think we can probably learn a few things about what happen
to Fidonet.
What (was) is Fidonet?
Fidonet was an early automated mail/file distributon standard popular during
the 80s and early 90s telecomputing modem and BBS systems days. Fidonet
allowed remote servers to 'network' themselves to help with the distribution
of mail and files around the world. It was inherently a modem based system
as the internet was yet available to the masses in the growing telecomputing
market. Your early BBS remote system were your early "Service Provider"
operation, in fact, many of the larger successful ISPs today started out as
BBS systems and as Fidonet Mail/File Hubs in the network.
As the internet became popular, Fidonet popularity began to dwindle but it
does still exist today among the purist, especially among for those who had
successfully merged internet methods with Fidonet methods. Our product is
one those early BBS mail/file systems that have evolved to successful
integrate Internet solutions.
In its pinnacle during the late 80, Fidonet membership grow to over 100,000
mail servers around the world with the majority US based. As the internet
grow during the 90s, I recall seeing an average NET lost of 50 mail servers
per week. I had calculated that by 2002, there would be less 5000 systems
representing the purist. Today, Europe holds the majority of the fidonet
network systems. This was because the cost of the internet access was
still an expensive endeavor in Europe vs the United States.
Fidonet offered a flexible network mail/file distribution topology with the
following addressing format per server in the network called a Fidonet node
address;
zone:net/node[.point]
Each member of the network was assigned a fidonet node address or number.
Where
A Zone was basically represented a continent, nation or country, i.e, 1
for US, 2 to 6 for parts of Europe and Asia, etc. See
http://winramturbo.com/fnsp/maps/world.htm for a global representation of
the zones in Fidonet.
A Net represented areas within a zone such as a state, province , city or
town or country, depending on many nodes were in the area. A fidonet
address with net number of 1 (net =1) was usually considered the main hub
or coordinator for the specific net.
A Node was basically 1 computer, 1 system using Fidonet compliant mail/file
server software.
For the most part, a Node was a "site" that offered online mail/file
services to connecting users. Today, this could be a web site.
For example, my old zone 1 fidonet address was 1:135/382. Net 135 covered
the Miami, Florida to Florida Keys area. I was node 382 in this net.
The optional point field represented an end-user for the node who wanted to
collect and send mail using Fidonet technology. A point was a compliant
fidonet node with no users but the point owner. This is akin to a user
with a mail client such as Outlook Express interfacing with his ISP only.
He was only allowed to send using his mail hub (ISP). This is akin to
whats going on more and more with the larger ISP systems restricting "users"
to send mail only using the ISP mail server. Using their own mail server
for direct SMTP operations is being blocked.
Having a point user was rare in Fidonet. It usually represented the more
sophiscated user who wanted to have the automated technology but really did
want to be in the business of serving users itself. It also represented
a group of corporate implementations who had mail/file hosting servers and
provided fidonet point software to their exclusive end-users for automated
Store and Forward system operations.
In my opinion, this "group of point users" in relationship to the internet
is probably the fastest growing group of potential mail systems for the
internet.
In Fidonet, an address with Net #1 or Node #1, usually meant it was a
central server or hub in a distribution chain such as a star topology.
node node
| |
node-----hub------hub---node
| |
node node
|
point
Fidonet Mail Distribution
Fidonet standards defined the management and process of network distribution
for both files and mail. File distribution is outside the whelm of my
discussion. I will concentrate on the mail, however, keep in mind that mail
is essentially files so the discussion here about client/server operations
is common to both.
In fidonet, there were two types of message or mail distributions:
- EchoMail
- NetMail
EchoMail is equivalent to today's newsgroups, and NetMail is equivalent to
today's email.
Echo Mail flowed from nodes to nets, net to net, and nets to zones and zones
to zones, etc. If a node subscribed to a particular echo forum
(Newsgroup), Fidonet offered a built-in automatic distribution logic so
that as mail flowed thru the backbone, echo mail was dropped off for
subscribing nodes in each net.
Netmail was a direct node to node connection concept pretty much like
today's direct smtp client to smtp server mail delivery logic.
Mail going to a destination address is looked up in a directory "phone book"
called the Fidonet Nodelist to begin the connection process and controlled
state machine conversation process, just like SMTP' MX concept.
Net routing was also part of the fidonet mail distribution attributes. It
was usually user defined or system controlled. For example:
To: Fred Address: 1:232/45
From: Tom Address: 1:178/5
[X] Send Direct
In this case, the mail was sent directly from 1:178/5 to 1:123/45. If net
routed, it was sent as follows:
1:178/5 to 1:178/1 Node #1 for Net 178 was considered the net hub
for that net.
1:178/1 to 1:232/1 Sent to destination Net hub
1:232/1 to 1:232/45 final destination
The same logic applied when crossing zones.
Often routed netmail piggyed back on the echo mail backbone distribution so
as echo mail flowed from hub to hub, any routed netmail was dropped off for
net delivery.
So how did this mail distribution and addressing system work?
The basis for Fidonet was its "NodeList" akin to today's DNS system.
The nodelist defined all members in the network and also defined the
capabilities and attributes of each system.
To be a "member"of Fidonet, your system software installation had to follow
two mininum FTSC (Fidonnet Technical Standards Committe) operational
requirements:
1) Foremost, Installed software must be compliant with the FTSC1
handshaking protocol, and
2) Each system must be available for mail delivery during ZMH (Zone Mail
Hour) and alway ACCEPT all mail from any system during this world wide
common mail hour period.
and that pretty much meant that it used the nodelist to decide how to send
or distribute mail and files, including the possible times a system was
available.
The above two requirements was probably the most heated debated issues that,
in my opinion, help with the demise of the Fidonet network.
The basic issue was that the FTSC1 protocols was outdated and kludgy. It
was based on a modified complex 7 bit XMODEM packet exchange handshaking
protocol. Developers were putting together new handshaking protocols and
state machine client/server conversations logics that addressed the growing
technical requirements, especially with the internet on the horizon. So
when new developers and software were coming on board, they often opted to
implement only the new protocols ignoring the FTSC1 protocol. And even if
they attempted to honor FTSC1, the complexity was so high, there were many
bugs and problems, thus violating the FTSC1 minimum requirement.
The second requirement of ZMH was in place so that there was always a
guarantee that a message can be delivered either directly or routed. This
was basically so that you can contact the person if there were problems. If
your system was not available during ZMH, it often meant you were
eventually expelled. Some of the hard nose administrators and network
coordinators who had a bug up their ass to enforce compliancy would run
automatic ZMH compliancy events to check each node ZMH availability in the
net.
While this requirement wasn't a big issue for developers, it probably
represented the early form of "unsolicated mail" operations since the ZMH
rule meant that your system must allow any address send you mail during this
special mail hour. The idea, of course, that you must allow "reports or
system oriented messages" be sent to you. This was particularily important
for first time operators. But it was a problem for many simply because
many didn't want just anyone to connect to them, sending unsolicited mail
and files.
These issues, to me, was probably the main reasons fidonet began to suffer
and lose membership.
Adminstrators bent on FTSC1 compliance would "excommunicate" members from
the network for non FTSC1 compliancy and if your software did not comply,
developers were shamed and not allowed to received a FTSC product code which
was used to identify the client/server software used a session. For many
new commercial vendors, it was hard to sell highly advance new software if
it was "labeled" as non-compliant with FTSC1.
Now yes, I can hear you say, "Thee Internet is different.".
I agree. The internet offered an open society which appealed to many of the
fidonet members. Those tired with the everyday working groups "Fight-o-Net"
arguments simply moved on to newer and more exciting times.
However, today, we are seeing the same growing "cries" from operators who
are sick and tired with spam. They are looking for solutions in almost the
same way old fidonet people were looking for newer and better solutions.
But allow me to continue with some of the technical aspects of the fidonet
network:
To become a member, you installed FTSC1 compliant software then applied for
a node number with your local area net coordinator (NC) who basically
assigned you the next empty or available node number for that net..
Comformancy testing was performed by the NC to verify compliant FTSC1
operability.
A NC weekly duty was to distribute a net change list file called the
"NodeDiff" file every friday to the zone coordinators (ZC). By the next
morning or day (saturday), a consolidated NodeDiff was created and
distributed to every member on the network world wide. Finally, the
Fidonet compliant software would recognized the new incoming nodediff file,
spawn a process to apply the diff file to the previous week nodelist
database.
Using the above netmail message example, if the destination address
1:232/45 did not exist in the nodelist, then the fidonet mail software would
not attempt to send the message. Each fidonet software vendor version had
it's own unique way of interacting with the user, but in short, if the
address was not in the nodelist, the mail was not deliverable.
Also, the nodelist offered many attributes that described a particular
system. In particular, attributes such as:
- No Direct Netmail
- ZMH (Zone Mail Hour) only
Another minimum membership requirement was that you were reachable one way
or another. If not directly, then via net route. If ZMH only, you could
not send any time of the day but only during the "Zone Mail Hour" or ZMH.
ZHM was usually 5 - 6 am GMT.
Using the above netmail message example, if the address 1:232/45 system was
identified as as no-direct mail system, it y meant you could only net route
the message. If the destination was ZMH only, you can only send mail
directly or indirectly during ZHM only.
There were other nodelist attributes that described the capabilities of a
system, such as:
- Maximum Modem Speeds,
- Possible file transfer protocols,
- Internet Ready System (connect address a IP address vs phone number)
etc.
When two fidonet servers connected, a followed a state machine concept that
is basically summarized as followed:
1) Client starts process by connecting to server sending a fidonet "packet
identifer"
The packet describes who the client is and what level of compliance it
supports.
2) Server verifies the client against the nodelist. Optionally, a server
can use caller id technology to compare the client IP or phone number in the
nodelist. If the client address is not in nodelist, or if there a IP/phone
number mismatch the server issues a negative packet and hangs up.
3) Once authorized, client sends all data files (mail plus files) assigned
to this server. It is possible that the client data files includes special
request for information.
4) Once the clients all its files, the roles are switched.
5) The server checks for any special request, if any, it is processed to
created the response.
6) The server sends all files it has for this client, including any special
request responses.
7) Once the server is complete, the session is complete and disconnected.
In short, that was how fidonet worked.
So what can we learn from Fidonet that can assist us in addressing the
problem of spammers in today's internet world?
Well, foremost, which is my primary concern, is to first recognize that
developers of SMTP servers are not administrators. Second, to recognize dev
elopers do have very good handle of the problem and the solution.
As with Fidonet, administrators who were enforcing the compliancy issues
restricted the progress and implementation of new ideas to solved and/or
address issues the system.
Now, again I fully understand the Internet is a more open system. RFC,
proposals and drafts are well structured and compliance among systems is
what makes the system work today. In addition, I fully understand the IETG
is not enforcing "rules" for "membership."
However, the mentality and effort to KEEP with the same standard is still
the same.
If we wish to finally address the spammer problem it will not be solved
using the current open, free for all outdated SMTP framework that we have
today.
The solution is based in LOCKING out the unauthorized system using enhanced
SMTP authorization solution.
Anything else, and we are just wasting time.
So whether its feasible or possible, I throw these ideas up for grabs:
1) Only compliant SMTP software should be allowed to distribute mail.
2) SMTP software vendors should registered to obtain SMTP product codes.
3) SMTP server DNS records should include attributes that better describe
the system, such as the following concepts:
- Product codes
- Send/Receive System
- Send Only System
- Receive Only System
- Private Only (Local domain only)
- Restricted Relay Allowed
- HTML allowed
- Attachments supported
- Bandwidth, T1, DSL, dialup, etc
- Open Hours
In summary, what I am proposing is that we concentrate on the
"authorization" aspects of smtp servers.
In my opinion, we really don't have a choice on this matter. We already see
it happening with more and more ISPs restricting residential systems.
This is the first shot of the bigger ISP systems organizing themselves in
such a way to create a "backbone" and expand the restriction list to include
commercial systems. Once one big ISP does something "different" that not
quite kolser with SMTP guidelines, others will follow.
So what about rDNS solutions?
rDNS proposals are probably among the best to enforce authorization concepts
and yet offer minimal software design requirements. We need to concentrate
on the best proposal, enhance it, agree and put it out there for developers
to begin using.
However, I believe it needs include concept I described above otherwise, we
are just beating a dead horse.
In my view, the problem is simple, it is well understood and there is only
one optimal solution - only authorized systems can be allowed to send mail.
Now, how about valid unsolicited mail (not spammers)?
The solution is simple.
All systems that want to be able to send unsolicated mail must "register"
with an organized registry system. This can be done during the installation
process of the software.
I will guarantee that if we had such a system in place, responsible
software vendors will include the "registration" logic or risk being locked
out.
How about Open Relay or Open Proxy problems?
Simple, again, part of the registration process may include the logic of
confirming the operability of your newly installed SMTP software. It can
test for open relay or open proxy operations as part of the qualification
process.
How about the politics, the "spirit of the internet" conflicts with such a
system in place?
Well, as far as the "spirit" of the internet, in lieu of the greedy aspects
of ISP using the "spamming problem" as an excuse to exclude the residential
systems, well, this will solve the problem. If the small "mom and pop"
system wants to have a small smtp system on his machine, it too must
register with the registry. In this case, the ISP has lost its excuse to
enforce static IP or commercial contracts.
As for the politics, well, the only thing I see is legacy systems or systems
with free software or abandoned software or what have you, with admins who
will always have a problem with "change" anyway.
The good news, if this system is put in place, it is of my strong opinion,
that there will be open software people who will write "wrapper" solutions
for legacy systems.
So even if people choose to keep existing software that works or they
already paid good dollar for and don't wish to "upgrade," etc, I'm sure
there will be developers who will write "proxies" or "wrappers" that helps
make their system work.
There you go. Please take it easy on me. I'm just a developer anyway where
simplicity of solutions are not always the most realistic approach to
problems. I prefer to work on a solution that will mandates compliancy of
smtp operations and lock out unauthorized and unsolicated mail distribution.
Thanks for taking the time to read this.
Hector Santos
WINSERVER "Wildcat! Interactive Net Server"
support: http://www.winserver.com
sales: http://www.santronics.com
_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg