Suppressing duplicates in the MUA is extremely difficult to do
properly,
to the degree that I would contend it's impossible to do with the
current
e-mail protocol in a way that would make sense to the casual user.
It's
necessary to suppress the personal copy, since the list copy is the one
that will sort correctly using the List-Id header and may have
additional
information or modifications made in the course of processing by the
mailing list management software. However, the personal copy almost
always arrives first, so doing this right requires temporary quarantine
and possible later release of mail, something that adds a tremendous
amount of complexity to mail handling.
I guess it depends what the purpose is in suppressing duplicates. I
suspect people don't want to see multiple copies of the same message in
the same folder. So when the personal copy arrives, no need to
quarantine it - just store it and index it by the message-id. When the
list copy arrives, realize that there's another message with the same
message-id, and link both of them to the same line in the message
summary. When the user clicks on that message, the MUA can say "there
are two slightly different copies of this message", do you want to see
the other one? (much like Google does when it identifies
nearly-identical documents at multiple sites).
If the message store is worried about conserving disk space, it can
just store the first message it receives along with the diffs between
that and the subsequent copies. (this works better if the message
store does it than if the MUA does it)
And yes, this only works if the message-id is properly maintained.
Most of the time, it is. But if the diffs are too great, or the list
alters the message-id, the message store can treat them as separate
messages. As long as this only happens occasionally I don't think it's
a horrible thing.
Keith