On Fri, 30 May 2003, Martin Stecher wrote:
I would use some very simple encoding to pass HTTP
messages in OCP payload. Something along these lines:
<chunk-type> <chunk-length> <chunk-data>
where "chunk-type" can be either of
"headers": HTTP headers including the first line and the last
CRLF terminator
"trailers": HTTP trailers including the last CRLF terminator
"body": raw HTTP message payload
"all": raw HTTP message
and one OCP payload may contain several chunks in the above format.
For a HTTP response we need the original HTTP request as additional
meta data. That needs to be added somehow.
Indeed, I forgot about that requirement. Will having both "request-*"
and "response-*" chunk types be too complicated?
request-headers, response-headers
request-trailers, response-trailers
request-body, response-body
request-all, response-all
(or the other way around: *-request and *-response).
We will also have to explicitly document what to do with 1xx HTTP/1.1
responses. It would be nice if we can adapt them, but I do not know
yet whether they have to be treated specially from HTTP binding point
of view.
While preprocessing is not in OCP scope, it should be o.k. to define
some requirements for application messages that may make some pre-
processing nececssary for some other types of the application
message.
I agree.
What I mean is: I think we can define "only identity transfer
encoding in this version of HTTP protocol binding of OCP" and so
force OPES processors to do some preprocessing for HTTP messages
that have a different transfer encoding.
Not saying that we should make such a rule. I am not 100% sure in
this moment. Could it be an option that is negotiated? Defining
which transfer encodings the callout server supports?
I think this MUST be negotiated, with a natural caveat that identity
encoding MUST be supported by all parties. We should make sure that
both chained and HTTP-unaware callout services can be supported
efficiently.
What I saw with ICAP implementations:
- Some ICAP servers forget to adjust the Content-Length header although
they change the body length. ICAP clients have a hard time then.
Can always happen, I guess. We cannot prevent that at a protocol level
unless we do not want to support persistency. However, a smart OCP
client can detect a OCP broken server, mark it as such, and switch to
non-persistent connections after the first broken response. Also, a
client may be manually configured to ignore server-sent Content-Length
(which would mean no HTTP/1.0 persistency as well, I guess).
- Often ICAP servers know that they are going to change something but
do not yet know when sending the HTTP response header back how big the
body will be. So, they need to delete the Content-Length header.
If this is a HTTP/1.0 message, the ICAP client needs to close the
connection; or it checks whether it can turn the message into a HTTP/1.1
message and adds chunked transfer encoding.
But most ICAP clients don't do this, so that ICAP servers started to
do the HTTP/1.1 transformation and add chunked encoding thing.
Big problem if the ICAP client does not support this because it is not
really HTTP/1.1 compliant.
Again, this seems to be a real-world problem without a general
solution because some agents are buggy or too assuming.
We can make the situation better if we explicitly negotiate who is
responsible for what. Defining one-size-fits-all solution (e.g.,
client is always responsible) seems too restrictive for me.
I wonder whether we could make things clearer and easier and defining
who is responsible for what.
We should.
What about adding a parameter to the app-message-start message that the
OPES callout server returns that has the message size (content-length)
if already known at that time.
Which is what "sizep" optional parameter in the draft is supposed to
do?
The OPES processor should be the only one being responsible to the
real HTTP message exchange with the HTTP client and can select
whether to close the connection, to keep it open, to change
Content-Length header or to add a transfer encoding. The OPES
processor does not need to trust the Content-Length header returned
but the value of the new app-message- start parameter, which if
present signals that the callout server knows what it does.
I agree.
This still does not solve all problems with buggy or assuming servers
that you mentioned above, right? It just helps in some cases.
Alex.