Re: Message Attributes (considered harmful)

> > it depends on what you mean by transferable.  surely a mail client
> > could use IMAP to move a message+annotations from one server to 
> > another
> > (as long as both source and destination servers supported that
> > feature), or from one folder to another.
> That's certainly easy to do. But what bothers me with this is the 
> prospect
> of having message (meta) data which can't be accessed for a variety of
> reasons. Perhaps the following is too simplistic or abstract:
>
> Currently the RFC 2822 format is flexible enough that a message can be
> considered a self-contained object. This is a very powerful property
> because messages can be moved around the world in many ways, not just
> the mail system.
yes.  this is a useful notion.  but it has its limits. clearly not 
everything belongs inside that container.    as the mail system becomes 
more sophisticated and supports more features, eventually a single 
container becomes too limited.
annotations seem like they belong outside the message container rather 
than inside.  there are cases where you want to copy them along with 
the message (say, when you're moving the message to another folder, or 
moving all of your mail to another server) but in the most common cases 
(forwarding, resending) it seems that you either don't want to include 
them at all, or you want them to end up in a different form on the 
destination end than on the source end (edited and/or stored 
differently to distinguish the original recipient's annotations from 
the forwarded recipient's annotations).
since you don't want the copy of annotations to be implicit in copying 
the message, I conclude that annotations should not be stored as part 
of that self-contained object.
(yes, you could require that every MUA edit out annotations from the 
message header before forwarding them, but this isn't backward 
compatible. it's also optimizing for the uncommon case)
> They can be moved by protocols such as SMTP, IMAP, POP3, they can be
> cut and pasted into applications, they can be imported/exported to and
> from servers, they can be stored or archived in mbox or maildir
> format in any filesystem. All of this is possible because a message is
> only a stream of bytes with some structure, and programs need not fully
> understand the structure to be able to do some useful things.
>
> Enter annotations, which would be meta information about a message,
> accessible via particular protocol extensions. Such information, if
> not embedded into the message, would be lost by the kinds of operations
> mentioned above. So the annotations would be second class information,
> stuck inside a given server, and only extracted with some effort.
>
> Perhaps that's what annotations ought to be? The last paragraph above
> is nearly beginning to look like a definition.
I don't think it's a definition, but I do think it approximately 
describes how they ought to work.  It's not that they inherently 
require more effort to access, because IMAP or POP commands to fetch 
both annotations and message might certainly be useful. The important 
features are (a) that they're associated with the message but not 
inherently part of the message and (b) that they can be manipulated in 
useful ways independently from the message.   For instance, you might 
want to allow a client to append an annotation, or to edit or delete a 
previous annotation.   The message wouldn't  change (say, for caching 
purposes) just because the annotation changed.
> > OTOH, it's not at all clear that annotations should be forwarded along
> > with a forwarded message.  certainly there are cases where this is not
> > appropriate.
> As I said above, I like the self-containedness of RFC 2822
> messages. It's well documented, both simple (if the MIME structure is
> ignored) and complex. Separate annotations would break this, unless
> annotations were considered value-less to most other software, as
> there would be some extra effort required to access them.
Actually I think that annotations have more value if they're not 
implicitly associated with a message than if they are implicitly 
associated with the message.
> But who's to say what has value in the long term?
we never know for sure.  but we make such judgments routinely when 
defining protocols.
(for that matter, who is to say what is long term?)

> Conversely, having annotations travel with the message (i.e. embedded
> somehow in the RFC 2822 format) will certainly encourage third party
> use of such information in the long run. So it will tend to add
> cruft.
>
> I don't know what I think of these options yet. I guess one way to 
> think
> of the question is this: if annotations exist as part of the message,
> then they do not impose new requirements on the existing transport
> infrastructure. While if annotations are intrinsically separate from
> the message, then the transport infrastructure must be aware of them,
> e.g IMAP needs an extension to access them, POP3 will need another
> extension, etc.
well, the client needs to be aware of them anyway.  otherwise they 
won't be visible, they'll get forwarded when you don't want them to get 
forwarded, etc.
imagine the case where a user uses two different MUAs to access a 
single message store, say one at home (a fancy IMAP client) and one at 
work (a webmail client, because the work firewall won't allow IMAP 
traffic to pass).  the fancy client supports annotations, the webmail 
client doesn't.  so the user annotates messages at home, but those 
annotations don't show up when he reads his mail from work - the 
webmail client is completely unaware of them.  the user doesn't know 
anything about the underlying protocol for how annotations work.  but 
since he only sees the annotations at home, he feels free to store 
private or sensitive information about messages as annotations.
so if the user decides that he's going to forward some message from the 
webmail client, should the webmail client send annotations along with 
that forwarded message or not?  if the annotations are stored in the 
message header, the webmail client will include them in the forwarded 
message (assuming it forwards as an attachment, which everybody should 
be doing these days), because it doesn't know any better.  if the 
annotations are stored out-of-band, the webmail client will not forward 
them, because it is not aware of them.  OTOH the fancy IMAP client can 
give the user the option of forwarding the annotations because it's 
explicitly aware of them.  for that matter it can let the user select 
which annotations to forward and which ones to keep private.
Keith