ietf-asrg
[Top] [All Lists]

RE: [Asrg] 2a. Analysis - Spam filled with words

2003-09-12 09:12:34
|-----Original Message-----
|From: asrg-admin(_at_)ietf(_dot_)org [mailto:asrg-admin(_at_)ietf(_dot_)org] 
On 
|Behalf Of Jon Kyme

<snip>

|> Perhaps a multidimensional Bayesian classifier could find these 
|> spam/ham
|> groups on it's own. Each method for bypassing filters in a 
|strange way
|> might
|> be easily discernable as a different cloud.

<snip>

|Good point, of course, many anti-spam content analysis type 
|things collapse measures made on a number of axes into one 
|metric. eg, spamassassin seems to turn the results of many 
|local and network tests into a single compact spam-score. This 
|is throwing away some information (?) Some way of preserving 
|the "depth" of the "test-space" might well give better 
|discrimination... I'd guess you're right.

There is a lot of heavy lifting involved when performing an analysis of
"strange attractors" - so real-time systems like filters cannot usually
afford the cpu budget involved. With Message Sniffer we solve this
problem by performing a rudamentary evaluation "in the wild" that
results in a single result that can be used by the filtering system -- 

Then, systems that wish to participate send back their log files which
contain the detailed datapoints developed from each scan. The systems
here can do heavy analysis on the features extracted from each message
and use that data to refine the rulebase. 

This is based on AI & real-time robotics research where a "reflex"
system capable of responding in real-time is continuously trained by a
slower, more sophisticated learning system.

A multidemensional Bayesian classifier would likely require a similar
division of labor to be practical - even if operated at the MUA.

_M

PS: In future we will divide the "heavy lifting" among the systems that
participate using distributed parallel processing techniques & cellular
automata mechanisms rather than having it done at a central location.


_______________________________________________
Asrg mailing list
Asrg(_at_)ietf(_dot_)org
https://www1.ietf.org/mailman/listinfo/asrg