Bad news for mailing lists archive site administrators.
Just doing a research on AltaVista, I surprisingly noticed
that the sites I'm currently administrating had been removed
from AltaVista indexes.
After reading AltaVista FAQ, it seems that they had
implemented some software which automatically removes every
site which have "a lot" of links pointing to the same URL.
They did it to avoid some sites to be overindexed within
their archives.
When archiving with MHonArc there is obviously as many links
to the Thread or Date Index 1st page as the number of messages.
It appears that it's too much for AltaVista.
I'd like to know if others had encountered this issue and what
did you do to fix it?
I've always intentionally kept message archives out of the reach
of search engines via /robots.txt (since excite didn't support
META headers last time I checked). Most subscribers to the lists
that are archived are thrilled with the extra privacy.
One possibility is to create an index page that concatenates messages
by day -- a sort of digest, if you will. That page could be indexed
by the search engines, then have links to the appropriate message(s)
in your MHonArc archives.
Chris
---------------------------------------------------------------------
Christopher Lindsey
System Engineer President / CEO
National Center for Supercomputing Mallorn Computing, Inc.
Applications http://www.mallorn.com/~lindsey
