mharc-users

Re: file permissions

2003-09-19 10:43:58
Earl Hood (earl(_at_)earlhood(_dot_)com) wrote:
One thing I'm doing is setting file permissions appropriately so that as
much as possible, public archives and search-related things are
other-readable (so the webserver can get them) and private archives are
not (and just served via mailman), which is the best compromise I can
come up with for protecting the privacy of list-members from nosy users
on my system.  The public-ness of the search stuff is far from perfect,
but I think at the moment I'm satisfied with protection from casual
poking-around, as it's better than what I had before.   This has mostly
involved little tweaks to web-archive.
  ...

I'm a little confused on the context.  Are you talking about
permissions wrt to the local file system or permissions as they relate
to web access?  I am guessing the latter.

Filesystem access, technically.  Though I'm applying permissions in some
cases that deny access to the webserver process except through the
already-suid Mailman cgi.

I'd like to keep nosy CGI-writers and shell-users out of private archives
without mucking around with suExec and any more setuid than I already
have to tolerate with Mailman.  Therefore, my goal is to funnel as much
access as possible through the Mailman cgi itself, at least in the case
of private archives.

If my site had *all* private archives, this would be *so* much easier.
Actually, if I had a dedicated mailserver with no unprivileged users,
I'd be long-since done.  Most of the gymnastics I'm doing are because
web, shell, and mail is all happening on one machine, and some of the
lists still want public archives.

If this is the case, mucking with file permissions is not needed
since all authentication should be controled via the web server.
I do not know much about Mailman, but it appears you could auto-create
htpasswd files (or dbms depending on which Apache auth module you are
using) from mailman data.  I.e. Create usernames to match subscriber
addresses and passwords to match subscriber passwords.

That's the solution I'd strongy prefer to avoid, though I do admit I see
its wisdom in the general case.  We've got over seventy mailing lists,
about two-thirds of them with private archives, with lots of users and
lots of add/drops and email address changes and password changes and
keeping the webserver password files in sync with the mailman passwords
requires a much better python hacker than I am to code hooks into
Mailman's user-management code.  Much, much better than I.  If I can make
this work on the mharc end, that means I'm working with perl, which means
I'm more likely to actually succeed & end up with something I understand
well-enough to maintain.

I think there are circumstances under which the solution  you've
described is the right one...but I'm not sure those circumstances are
mine.  I admit I have a perhaps over-specific idea of what results I'm
looking for and how much time it'll take to administer.

Someday, someone in the Mailman community will write a simple,
generalized interface to the Mailan authentication procedure and I can
throw all this away.  Or I'll improve my python skills to the point
where I can write it myself.  But I do not advise anyone to hold their
breath; I've been thinking about that for several years and it's never
made it high enough on my priority list. Or, apparently, anyone else's
(but if I'm wrong, please tell me -- I would be delighted to find a
simpler solution, even with as much fun as this has been).

Searching is a bit tricky since the same CGI program is used for
all archives.  Therefore, someone could technically perform searches
on private archives (but they would still be unable to access entire
messages).

Yes.  I'm accepting that; potentially exposing message-fragments is at
the moment an acceptable trade-off for the search functionality.  If one
of my listowners decides that's not the case, my next project will be to
figure out how to disable search on a list-by-list basis.  I'm very much
hoping I won't have to do that.

The extract original message CGI has the same issue, but it
can be used to circumvent any web access permissions if someone
knows the message-id of the message they want.  It does seem
if someone knows the message-id, than they already have a copy
of the message.  But they could share that information (by
a direct link) for others to bypass authentication.

Whoo.  I didn't realize that.  I will examine that functionality as a
candidate for disabling, then.

My question is:  I'm thinking about hacking a chmod into web-archives
&namazu_cleanup(); is there perhaps a better place for me to put this
code?  I don't think I want to change mknmz so its behavior is changed
for all users; other people on my system are already talking about
wanting to use it.  Or am I misunderstanding why my permissions on those
files are coming out they way they are in the first place (in which case,
I can take this question to namazu-users-en instead)?

Not knowing all that you are trying to do and how you have things setup,
I am concerned if you are implementing the best solution for your problem.

Oh, probably not, for any number of values of 'best'. :)  Much of the
logic behind my tweaks was driven by my preference for making local
changes to mharc, not only because I'm more comfortable with perl but
also because mharc's architecture seems better-suited to local
customization, whereas Mailman falls over when I look at it sideways.
I'm also looking for a solution which requires the absolute minimum of
intercession on the part of my listowners and list-subscribers, many of
whom are nontechnical and/or easily confused.

I'm fairly certain that a more-skilled-than-I Python hacker could have
made changes to Mailman to support better integration between
password-protection and external archiving.   However, the Mailman folks
have publicly stated that their focus is on public archives & therefore
I'm not looking for much support from them in this endeavor.  I think
their focus makes sense; the vast majority of their userbase appears to
be people running technical mailing lists, in which public access to list
archives means a smaller mailing list can support a vaster userbase.  My
users, however, are largely doing artistic and social discussion and
they're much more panicky about their privacy.

(final step: appropriate permissions for the mboxes.  If I'm not
mistaken, I just have to make sure that the folder created to hold a
lists's mboxes has the right permissions, and procmail will do the right
thing with regards to creating new yyyy-mm mboxes.  and yes, I realize
this means that my listowners will have to contact me if they want to
change their lists public/private archive status.  I've let them know
of this & nobody's complained yet, so I'm keeping my fingers crossed.)

I guess it looks like what you are doing is particular with mailman, so
I can only provide little guidance there.  Whatever the issue, attempts
should be made to keep mharc and mailman as separate as possible (i.e.
mharc should not be making any mailman specific calls).  However,
mharc could be updated to include features that would facilitate
such integration (like options to specify file permissions).

I'm not sure my perl changes are good candidates for integrating into the
general mharc distribution.  Most of them are pretty mailman-specific.
The .mrc changes I made to make the archives look more like the
pipermail-generated ones might be interesting in a wider context; beyond
that, the gotchas I documented I extracted from the mail archives for
this list in the first place.  

I generalized all of my changes with an eye towards sharing my
experiences with other site-administrators, not with the development
community at large (which is why I'm here and not on the developer's
list, though if you really think I'll be happier over there even with
that caveat, I will take that advice).

If you have the desire to read 76k worth of "here's what I did", it's at
http://www.byz.org/~sev/Tech/integrating-mharc-and-mailman.html
...the part about modes on search indexes isn't actually working at this
point, but I'll fix the diffs in that document when I figure out what I
did wrong.  Everything else works as advertised as far as I've been able
to test.

The section specifically on filesystem permissions is here:
http://www.byz.org/~sev/Tech/integrating-mharc-and-mailman.html#changeperms
but I'm not sure how much sense it makes out of context of the rest of
the document.

I'm also not sure how much sense any of it will make to someone not
already wrist-deep in Mailman administration.  I've tried to explain the
'why' behind my changes but I'm not up to actually explaining how
Mailman works.
 
BTW, you may want to contact the Savannah guys about integration of
mailman and mharc.  The GNU folks use mailman as the list manager
for savannah projects, but they use mharc for the mail archives.
I have not received any message from them on the nature of what
they actually have done, but they may have done some of things you
are doing.

Thanks for the pointer.  I've seen their basic description of what they
did and it looks like they're not addressing private archives, which is
what I'm focusing on this week.  Their and others experiences integrating
Mailman's public archive-access with mharc have been very useful (thus
proving once again why public archives on technical mailing lists is a
good idea!), and I'm very glad I was able to learn from it -- and now I'm
trying to build on it.

(which, technically, is why I'm writing up my changes in such detail;
I've benefited so much from other people's experiences that I'd like to
give back if I can.)

sev

-- 
 *** sev(_at_)byz(_dot_)org can also be found at http://www.byz.org/~sev ***
     I <heart> my rotating .signatures

---------------------------------------------------------------------
To sign-off this list, send email to majordomo(_at_)mhonarc(_dot_)org with the
message text UNSUBSCRIBE MHARC-USERS