procmail
[Top] [All Lists]

Messing with E-mail Headers

1996-08-09 13:23:48
    > Thanks for your help, but I tried doing that - I end up getting
    > the email correctly, but the headers don't seem to be formed
    > properly:

    > The first From is smmi, which is the user that is getting the email, the 
    > second From (which differs from the first) is the correct one. Any ideas 
    > on how to remedy this?
    > 
    > 
    >  Message 97:
    > >From smmi Sat Aug 10 02:55 KST 1996
    > >Return-Path: micah(_at_)smmedia(_dot_)com
    > Date: Fri, 9 Aug 1996 10:56:35 -0700 (PDT)
    > From: Micah Anderson <micah(_at_)smmedia(_dot_)com>
    > To: smmi

You are now describing a different problem.  Your original mail said
that you were NOT getting the e-mail.  Now you say that you *are*
getting the e-mail, but not "correctly".

Since there seems to be some confusion, at least on my part, and since
it appears that we may not agree on what is "correct", I'll start with
the basics (from an earlier composed explanation); if you already
understand most or all of these basics, then great, skip ahead.

============================= cut here ===================================
E-mail Headers

There are two general classes of headers: those generated automatically
by the MTA, and those configured and inserted by the MUA, on the user's
behalf.  

The former, the ones generated by the MTAs, are used mostly for tracking
the e-mail, and generally have nothing to do with the content of the
email, much like those bar-code labels FedEx uses to track packages.

The latter, the ones inserted by the MUA or by the user, are just like
the shipping label the FedEx customer fills out, ie: they determine the
source, the destination, and describe the content of the mail.

It would be overburdensome for the user to generate all of these MUA
headers themselves, so the user's mailer generates many or most of them
automatically, typically under configration control.  Of course, the
user can always override or replace the automatic MUA headers.

The MTA headers, on the other hand, are almost completely automatic and
the user almost never can change them.  Only under special circumstances
should the MTA headers be inserted or modified by the user.

From the user's pespective, however, the e-mail process seems atomic, so
that the distinction of these header classes is lost.  Even some systems
managers or postmasters fail to appreciate that it is during different
stages of the e-mail process, that different sets of headers get
inserted.

To help clarify this distinction, here's a diagram of the e-mail process
and its several stages:

 sender -> MUA -> MTA -> ... -> MTA -> MDA ->{maildrop}-> MUA -> reader
        [1]    [2]       [3]        [4]    [5]                [6]

Headers typically provided by "template" by the MUA to the sender,
usually during stage [1] (when composing e-mail):

    From:               # who I am
    To:                 # the target
    Cc:                 # people to keep informed, but need not respond
    Bcc:                # secret admireres
    Subject:            # what's the mail about
    Reply-To:           # highest priority return address
    Priority:
    Precedence:
    Resent-To:          # used for redirecting e-mail 
    Resent-Cc:
    X-BlahBlah:         # personalized headers

When the sender is done composing, and says "send it" to his/her mailer,
some additional headers may get inserted by the MUA at this stage [2]:

    Date:
    Resent-Date:        # if being redirected
    From:               # If not already present
    Sender:             # if a From: is already present
    X-Mailer:           # what MUA composed this message
    Mime-Version:
    Content-Type:       # what kind of stuff is in here
    Content-Transfer-Encoding:
    Content-Length:

When the MTA receives the e-mail from the MUA at stage [3], it may
insert additional headers showing the origination of the e-mail:

    From                # if local e-mail, automatic or by -f option
    Date                # If not already present
    Message-Id:         # unique ID for the e-mail; the first MTA
                        # creates this
    Received:           # shows inter-system e-mail tracking info
    Return-Path:        # shows how to get back to the sender

As each MTA hands off the e-mail, additional headers may get added, all
as part of the MTA to MTA handoff in stage [3]:

    Received:           # inserted by each MTA

As the final MTA hands the e-mail to a delivery agent (MDA), in stage
[4], there are still some more header insertions which may occur:

    Apparently-To:      # added if no To: header exists
    From                # may get added if local e-mail

Some sites insert special rewrite rules and filtering to occur to
support virtual domains, and these header changes will occur at stage
[5], just before the incoming mail is dropped.  Generally, though, no
new headers are added, except possibly one to avoid loops:

    X-Loop: $USER(_at_)$HOST    # inserted to avoid filtering loops

Finally, at stage [6] when the reader views his/her e-mail, most MUAs
will apply a filter to the stored mail causing selected headers to be
omitted from the display.  In a sense, then, this filtering "removes"
the headres from the user's view (although no headers are actually
removed by the MUA).

The headers typically omitted are those inserted by the MTAs, and those
having to do with the transport process and less with the contents.

...

============================= cut here ===================================

So, now that we have a common understanding...

The first "From" is a Unix-mail "From " header (note the space).  This is
inserted automatically by MTAs, unless one is already present and only
then if it seems valid.

The second "From:" is generated by the MUA (your personal mailer),
either by configuration, or by the user.  The rewrite rules in sendmail
and most filtering programs concern themselves with the "From:", "To:",
"Cc:", "Reply-To:" headers.

I'll assume that if "From smmi" is not "correct", then you must be trying
to hide the delivery process, and implementing something of a virtual
domain. 

In general, it is a bad idea to "correct" the automatic mail headers
inserted by the MTAs.  This is a different matter than changing
addresses to show virtual domains.  The "From " header is part of the
history of the message, showing how the mail was originated.  Similarly,
the "Received:" headers should not be messed with.  Changing the history
of an e-mail message will make it very difficult to diagnose e-mail
delivery errors.

That being said, and, since I also believe in the freedom of choice, I
will now supply you with "enough rope to hang yourself" :^)

There are two places where you can have the "From " header corrected:
just before it gets dropped into the mailbox (for incoming e-mail), or
as it gets submitted to the MTA (for outgoing e-mail).

Changing the "From " before it gets dropped is easy.  Just use a recipe
like this:

    FROM=`formail -zxFrom:`
    DATE=
    :0fh
    |formail -I"From  $FROM `date`"

The "From " header is created automatically by the MTA (sendmail) when
it receives a piece of mail.  If the mail is sent through sendmail
without using the '-f' option, then sendmail sets the default "From " to
that of the current user.  If you are not root, or a "trusted user" (see
the sendmail man page), then sendmail will ignore the "From " header and
either remove it altogether or replace it.  Even if you are root,
sendmail will replace the "From ", if the e-mail is being received
locally (as opposed to from the network).

If you wish to change the "From ", you must invoke sendmail, as root or
a "trusted user", and use the "-f" option.  EG: to set the "From " to
match the "From:" header, use the following recipe, as root:

    FROM=`formail -zxFrom:`
    :0
    ! -oi -t -f"$FROM"

Please read the man page on sendmail, noting the use of '-f'.

G'luck.

_____________________________________________________________________
Alan Stebbens           <stebbens(_at_)sgi(_dot_)com>             (415) 933-6437
Digital TV, Silicon Interactive Group,  Silicon Graphics, Inc.  (SGI)
M/S:9L991,     2011 N. Shoreline Blvd.,     Mountain View, CA   94043
PGP Key Fingerprint: 94 A7 4B AB 1C F0 4D 92  DD BC B5 D7 11 A0 DC B3

<Prev in Thread] Current Thread [Next in Thread>