nmh-workers
[Top] [All Lists]

Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)

2008-10-22 22:32:50
Mark wrote:

In the message dated: Wed, 22 Oct 2008 09:00:12 BST,
The pithy ruminations from Peter Maydell on
<Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)> were:
=> David Levine wrote:
=> >Mark wrote:
=> >> Peter Maydell wrote:
=> >> => I think this has been reported on the bug tracking system before.
=> >> => (https://savannah.nongnu.org/bugs/?14975)
=> >>
=> >> Yes, that does sound like the issue I'm seeing. Note that the bug
=> >> was opened in 2005. I'm really not a "C" programmer, but it took me
=> >> almost as long to read the bug report as it did to increase the
=> >> value of NAMESZ. As far as I'm concerned, this is a trivial fix to a
=> >> signficant interface issue (ie., not being able to see the fields in
=> >> the scan listing).
=>
=> Yes, but I'm not sure it's the right fix. Most of that bug is me trying

Philosophically, it may not be the "right" fix. Practically, it fixes the
problem with scan.

Perhaps the "right" fix is to simply ignore lines in a mail file that preceed
the body text but are not valid headers. As per RFC 1122:

      "Be liberal in what you accept, and conservative in what you send."

This seems to be the current behavior, as long as the invalid line is less
than 128 characters.

When scan encounters a "From " line (note the missing colon), scan
is happy to accept it, unless it's over 127 characters...in which
case the complaint is about the length, implying that there's
nothing wrong with a "From " line in the header block.

=> to get information from the guy who reported it. (It didn't help that the
=> file he attached as a test case had got mangled in transit somehow so that
=> it started not "From ..." but ">From ...".)
=>
=> >> => in a mail spool file and so get stripped out as the messages are
=> >> => split). So perhaps this bug should be fixed by making whatever
=> >> => path you're using to put messages in the folder strip out the
=> >> => envelope From as inc does.
=> >
=> >Ah, I had procmail configured to insert the 'From ' line,
=> >with its -f option.
=>
=> Well, er, don't do that then.

I don't any more :-]

=> The mh-mail(5) manpage is pretty clear about what the format of files
=> in an nmh mail directory ought to be. These 'From ' lines simply aren't

Actually, the man page states:

      It should be noted that although neither Bell nor Berkeley mailers
      produce message files in the format that  nmh  prefers,  nmh  can
      read message files in that antiquated format.

That "antiquated format", as far as I can tell, is the mbox
specification, where messages begin with a "From " line. Thus, my
reading of the mh-mail manpage is that nmh doesn't necessarily use
the "From " line, doesn't write a "From " line, but that it should
accept a "From " line. In fact, it _does_ accept a "From " line
(thus, there's no need to try to track down every program in the
universe that creates or doesn't remove an existing "From " line and
provide instructions for how to make them compatible with nmh). The
problem seems to be that nmh just doesn't handle "From " lines (and,
I suspect, other headers) longer than 128 characters.

=> valid within it.
=>
=> If the code writing 'From ' lines is part of nmh we should fix it;
=> if it's third-party then we should probably identify and document
=> how to configure that code to write valid nmh files.

How about adding this to docs/MAIL.FILTERING after the current
paragraphs that discuss the "From " line:

  Alternatively, you might be able to suppress generation of the "From "
  line.  If your procmail invocation includes the -f or -r option, try
  removing it.  Those options add "From " lines to the beginning of
  incoming messages that do not have them.

=> >> Perhaps, as the bug report from 2005 stated, that would break mail
=> >> filtering that actually looks at the "From " line. Why strip it
=> >> out...it's data about the mail delivery process, and may have some
=> >> value to some other process (spam filtering? mail statistics? random
=> >> number generation?).
=>
=> If you want the information you should write it in as an X-Envelope-From:
=> header or similar.

Maybe. Some programs (anything that deals with the mbox format, for example)
expect "From ", not "X-Envelope-From:".

=>
=> >> Personally, I think that the philosophical argument about whether
=> >> tools strip out the "From " line or not is really separate from the
=> >> fact that the the scan listing breaks because of an arbitrarily
=> >> small value in NAMESZ. The first issue is difficult to resolve (as
=> >> evidenced by the 3 year-old bug), while the second issue is trivial
=> >> to fix (though maybe not as "pure" as some would want)....but what
=> >> do I know? :)
=> >
=> >I agree.  Any objection to raising NAMESZ to 512?
=>
=> It's still an arbitrary limit. I'd be more interested in raising
=> it if there were actually problems because of messages with header
=> components with names longer than 128. (The 'From ' lines don't
=> count because they're not valid syntax.)

Again, the mh-mail manpage states that nmh will read messages that contain
those lines, even if they aren't used by nmh, so they are not defacto invalid
syntax to nmh.

=>
=> (If you do raise it then my reading of RFC2822 says 998 would
=> be more defensible than 512.)

Perhaps the mh-mail man page needs to be updated. My mh-mail
manpage, from the 1.3 source, cites a limit of 63 characters, as per
RFC822.

That's different, that's for header names.  That should say 126 now
(excluding the :, which is included in the 127 bytes in the buffer).

The 998 is the limit on the length, excluding the trailing CR/LF, of any
line.

In any case, raising the limit does seem to be the answer...

Agreed.  I don't think that dynamic buffer allocation is worth
the trouble here.  We're just trying to avoid a harmless warning.

David


Mark "feeling pedantic this morning" Bergman

=>
=> -- PMM
=>

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________


_______________________________________________
Nmh-workers mailing list
Nmh-workers(_at_)nongnu(_dot_)org
http://lists.nongnu.org/mailman/listinfo/nmh-workers

<Prev in Thread] Current Thread [Next in Thread>