procmail
[Top] [All Lists]

Re: need a sane approach to MIME attachment organization

2012-10-24 01:29:07
Mike's unattended mail spake on Tuesday 23-Oct-2012@11:11:17
On 2012-10-21, LuKreme <kremels(_at_)kreme(_dot_)com> wrote:
* you lose the ability to automatically process a file attachment
  and carry out realtime actions on that file the spot (unless you
  keep your MUA running 24/7 and code it to do the job of procmail).

Actually, you don’t. Processing the file does not necessitate
stripping the file.

Sure, but solving one problem causes another.  Either you lose the
ability to automate the management of your files, or you lose the
ability to avoid redundant copies.  An inherently flawed structure
forces you to choose a problem.  Theoretically, there is no reason we
can't have it all.

To my mind, the original message stays intact, period. Anything you
do after that should never change the messages that was received.

Why?  You seem to lack confidence that automated tools could handle
this, no?  Is a dedicated special purpose tool more likely to corrupt
a file than a fully featured MUA?

There are many issues here. The primary one is that the original email should 
always be recoverable. The original email should always be archivable. The 
original message should also be forward-able.

For example, let's say I have an IMAP account address of 
kreme(_at_)domain1(_dot_)tld and I change my ISP and have a new IMAP account 
address of kreme(_at_)domain2(_dot_)tld.

If the messages are not destroyed by the MTA/Procmail then to transfer all my 
mail archives from domain1 to domain2 I simply drag the folders from one iMAP 
store to the other. With your method, it is impossible, because the original 
message has been destroyed.

Duplication is not just a waste of space.  There are update anomalies.

No there aren't because the original email always contains the original 
document.

It's generally poor management of data to have multiple copies running
around.

Nonsense. That's exactly what a backup is.

 When you need to destroy something, you potentially have to
find two copies to delete.. or just one, because you won't necessarily
know where the file came from or if it's replicated.  And when you
update the metadata on an external file, the mailed copy doesn't get
the update.

That's right, because the mail is untouched.

Now suppose I accept your idea that there should be two copies, one in
the email and a proper file that exists independantly on a filesystem.

But you're already showing a prejudice; there are not "two copies" there is the 
original email, and there is the document independent of the email.

What links or associates the proper file to the email containing the
duplicate attachment?  ATM, I've found no tools that make it trivial
to jump between the two.

Like I said, finding an attachment in my MUA is trivial. They are not linked, 
nor should they be linked, any more than a backup from last month should 
reflect the changes you made to the document last week.

I would be all in favor of dumping mbox for a format that maintains a
relationship between the message and the file.  But I would not be
keen to replace mbox with something else that also does not adequately
establish links and metadata between the message and external content.

Suggestions?

No, because in my opinion you are looking at it completely wrong.

If the original message with the original content is not useful, then the user 
can delete it.

I'll give another example.  Having the file encoded in base64 ASCII,
and embedded in a message, is a lousy way to store a file.

But it's an excellent way to store a MESSAGE, and that's what I see; not a 
file, but an email.

Consider having a need to search for text or some object embedded within your 
files. Your search tool must then be aware of your email archive format, and 
it must extract each attachment, decode it, and search it. It's CPU 
intensive, and human intensive as well if files are embedded in encrypted 
messages.

So you might need to go back to the original message to get the original 
attachment? Or not? Because you appear to be saying both things here.

* the file must be extracted and potentially decrypted every single
  time it is opened from the message (if not duplicated).

This is a good thing.

Not if you've forgotten the password that decrypts the private key
needed to decrypt the file.  This has actually happened to me, btw.
And keys expire.

If you extracted the file originally, why would *you* ever need to go back to 
the message to re-extract the file? You are specifically asking for a way to 
prevent this automatically, aren't you?

If the message is not valuable as it is, delete it.

Hold on.  A /search/ is one way to find things, and
browsing/navigation is another.  Having a good search capability is
only half of it.  If you optimize your organization for browsing
attachments, you compromise on browsing for messages, and vice-versa.

No, that doesn't follow at all.

procmail was written long before MIME email became common.

Does this mean the language (and thus tool) does not evolve?

procmail has not received significant updates since the late 90's. The current 
version, featuring a few tweaks and bug fixes, is from 2001. I would say that 
in computer terms procmail is definitely not evolved, and practically Jurassic.

If I wanted to join the project to expand the language and add some basic
features, would this be welcome?

Welcome? Yes. Problematic? Probably also yes. I do not know who 'owns' the repo 
at this point. You would probably need to fork the current source and create 
your own repo. So far, no one has taken up the task of updating procmail, 
despite various threats. For those of us using it, it's good enough for what we 
need to do. If it falls down too much for what we need, then we look at 
something like maildrop.

Maildrop is especially good when using single-file-per-message stores like 
Maildir.

That said, MIME and attachment handling is problematic, period.

-- 
When treading water in a circle of sharks, a wizard will always consider
other wizards to be the most immediate danger. --The Last Continent


____________________________________________________________
procmail mailing list   Procmail homepage: http://www.procmail.org/
procmail(_at_)lists(_dot_)RWTH-Aachen(_dot_)de
http://mailman.rwth-aachen.de/mailman/listinfo/procmail