ietf-openproxy
[Top] [All Lists]

RE: transfer- and content-encoding

2003-10-13 03:23:40

Hi Alex,

Do we need to prohibit encoding sequences like
(chunked,identity,chunked,identity)? The above rules do not.


Do you mean multiple usage of the same encoding in the list of the
Transfer-Encoding member?

No. Repetitions in Transfer-Encodings list should not cause any
problems.

Or that a transfer encoding could be applied several times 
to some data?

Right, the same transfer encoding(s) applied several times.

      Transfer-Encoding: (chunked,chunked,chunked)

What kind of problem do you see here?

If we say nothing about it, them existing rules imply that the
recipient should handle (chunked,chunked,chunked) encoding if it
advertised support for chunked. I doubt many implementations would be
that flexible; I doubt they would be able to dechunk the same content
three times. It might also be a security vulnerability for the service
if the list gets rather long or something of that kind. That makes me
think that we should say something about it, if not outright prohibit
it.

RFC 2616 has some text related to this problem, though its motivation
is probably different:

   When the "chunked" transfer-
   coding is used, it MUST be the last transfer-coding applied to the
   message-body. The "chunked" transfer-coding MUST NOT be applied
   more than once to a message-body.

Should we copy that?

I like your general approach below even better.


It is unlikely that OPES processors would _add_ transfer encoding
because there is really no need for it with OCP. Content-encoding can
be used for compression, I guess. So we are most likely talking about
removing chunking or any extension encoding supported by the
processor.

I agree.


I would suggest that we require identity transfer encoding, but I am
afraid that some processors may not want to dechunk a message (perhaps
they are doing some kind of zero-copy I/O or TCP slicing?). Another
problem is that if there is an extension transfer encoding than the
processor may not be able to dechunk back to identity.

It's a mess.

So, how about a SHOULD? Agents SHOULD support both identity and chunked
transfer encoding.


How about this:

      If OPES processor applies transfer encoding, it MUST NOT
      apply any encoding more than once. Callout services
      MAY reject content for which a transfer encoding
      was applied more than once (XXX: document this as
      a potential exploit area).

Good, I just want to generalize to agent and peer rather than OPES
processor and callout services. Also the callout server MUST NOT
apply any encoding more than once.


And then perhaps give a typical example with chunked encoding being
stripped by the processor.


I now wrote this paragraph.

2.2.3 Transfer-Encodings

   The agents need to negotiate their encoding and decoding
   capabilities. Transfer encodings that are not supported by one agents
   MUST NOT be used. HTTP messages with a transfer encoding that is not
   supported by the callout server MUST be decoded by the OPES processor
   and eventually be re-encoded with a supported transfer coding.

   Negotiation is done using the Transfer-Encoding parameter of the
   profile feature. Both agents advertise a list of supported
   transfer-codings for encoding and decoding. Encodings listed earlier
   are preferred. The Transfer-Encodings parameter MAY be omitted if
   only the identity coding is supported.

   An OCP agent receiving application data from its peer, MUST be ready
   to accept multiple transport encodings, in any order, if it
   advertised support for multiple transport encodings. If an agent
   applies transfer encoding, it MUST NOT apply any encoding more than
   once. The peer MAY reject content for which a transfer encoding was
   applied more than once (XXX: document this as a potential exploit
   area).

   OCP agents supporting HTTP profiles SHOULD support and advertise
   identity and chunked transfer coding.

                Transfer-Encodings  = "Transfer-Encodings:" SP
                                      "(" [transfer-coding-list] ")"
                transfer-coding-list = transfer-coding *("," transfer-coding)

                for example
                        Transfer-Encodings: (identity,chunked)

                                Figure 5

   transfer-coding is defined in section 3.6 of [RFC2616].



What do you think?

Regards
Martin