ietf-mta-filters
[Top] [All Lists]

Re: 'header' test and whitespace

2005-11-21 18:07:59

First of all, my apologies to the list for failing to get out the
updated 3028bis I-D by last Friday as I promised.

Haven't gotten the revised vacation draft out either ;-(

Part of the hold up has been in determining the exact wording to
insert on a couple points.  In particular, Barry Leiba argued
strongly in Vancouver that the 'header' test should _at_least_
normalize to a single space all whitespace that appears at the
beginning or end of each line of a header field value.  That is,
the header field unfolding algorithm would not only remove the CRLF
pair but also compress the whitespace around the CRLF into a single
space.

Alternatively, that same compression could be applied to *all* the
embedded whitespace in the field value, such that the 'header' test
would never 'see' tabs or consecutive spaces.

So, there are three choices:
1) just compress whitespace at fold points
2) compress all whitespace
3) no compression

I strongly object to (1) and (2). Making the header test not respect interior
spaces is a _really_ bad idea. Like it or not, there are communities that are
very sensitive to spacing issues (many Japanese users seem to qualify) and
these communites need to be able to test and see exactly what sorts of spaces
are present in fields (and possibly modify them).

Additionally, compression down to a single space is actually insufficient in
some cases - what's needed is for spaces to be removed, not compressed. But
this can, and should, be done through an appropriately defined comparator. It
should not be a built in part of the header test.

Note that we had previously agreed that leading and trailing
whitespace on the field value in total should be completely ignored;
that would not be affected by the compression described above.

I have no problem with removing whitespace at the beginning of the field - in
fact I prefer it. I think removing trailing whitespace has a chance of causing
problems, but the benefits probably equal if not exceed the costs.

                                Ned