Martin,
I think you've raised two important design questions that must
be addressed explicitly, possibly in OCP Core (in part).
(1) We need to be more clear what is being adapted, what is
being supplied for information purposes only, what is being skipped,
what is potentially stale and must be checked by processor (and when
the checks are made).
(2) We need a "content" size prediction for performance
reasons, but we do not have a definition of "content", only data.
Sizep seems to be useless. A new parameter, tied to the "content"
definition is needed. "Content" definition should probably be
application-specific and negotiable.
On Fri, 17 Oct 2003, Martin Stecher wrote:
it is not even that the OPES processor MUST NOT by default trust the
Content-Length header returned by a callout service, actually
neither agent can trust that header.
Debatable. This depends on where we place processor header-adjustment
requirements: after all processing or after each processing step.
An OPES processor may call a first callout service and then forward
the response to a second callout service, i.e. implementing a pipe
of callout services.
True. A good design will not place the second callout service at a
disadvantage. Thus, either all callout services should assume
out-of-sync headers or processor should fix headers after each
service. The former deprives certain services from required
information (they would have to buffer whole responses to obtain it).
The latter makes pipelining impossible or inefficient in some
situations.
We should not force the OPES processor to do the header correctness
check and recalculation after each callout service response but only
at the very end before sending on the HTTP message on the HTTP path.
That's a design choice, but I tend to agree with your preference
because more systems will benefit from a pipeline than from having
message length to be known in advance. However, I sense that we are
facing a bigger problem here that has to be addressed on a higher
level. See below.
That means that a callout server may see the HTTP message that was
already modified by another service and hence some headers may be
incorrect.
Exactly. Essentially, all headers in this case represent potentially
stale information and are subject to further modification by processor
and other services. On the other hand, a service may want to adapt
some headers. Thus, we have one object (headers) that is partially
stale and partially adapted. This is a mess.
I would therefore like to require that both agents MUST use sizep
parameter if they know the message length and no agent SHOULD trust
the Content-Length header.
That would be a logical conclusion, yes. In that case, I would want to
argue that Content-Length HTTP header MUST NOT be transmitted over
OCP. However, logging services would need complete headers so that is
not acceptable.
That makes it also simpler for the callout service to use the sizep
parameter. If the service does not need to parse the HTTP headers
for its filtering purpose, it would not be nice if it needs to
lookup the content length header value in order to insert sizep
parameters. If it already receives sizep from the OCP client, it has
an easier job.
Agreed.
The OCP client, as typically being tightly bound to a HTTP proxy,
has probably easy access to the content length header of the message
(if any).
Agreed. Even if the bound is not tight, a client must know message
size one way or the other.
On Fri, 17 Oct 2003, Martin Stecher wrote:
The sizep parameter is the "remaining application data prediction"
according to OCP-Core draft.
I see some conflicts:
1. What if the agent knows that it will skip certain message parts?
Does it need to substract this skipped size from the sizep
parameter? This will break our Content-Length idea.
sizep should refer to what the after-adaptation application message
will look like. If a service deletes parts of a message, sizep will
decrease. If a service ignores parts of a message but expects
processor to add them, sizep will not change. Moreover, only
content-related parts should be counted.
Or does skipping of parts not affect sizep parameter? Is this then
still the "remaining application data"?
It looks like the definition has to be polished to reflect the "OCP
client and server are working together on a complete application
message, whether they see/skip/ignore parts or not" view. The
definition of "application message" is to be negotiated. More below.
2. What about optional parts? Should their size be added to sizep?
Will again break Content-Length trick.
We need to change something here. Sizep should be referring to the
thing we are adapting, not optional header information that may be
supplied. Right?
3. Trailers belong to the application data. Therefore the DUM
message containing the response-header should have a sizep parameter
equal to response-body size plus response-trailer size. But this is
not the size that we can use for Content-Length correction.
True. It looks like sizep does not work on application data level (at
that level all data looks the same). It works on a higher level, where
headers are different from bodies different from trailers. I do not
know if we can document sizep in a application-agnostic way.
4. If response-header part is sent in multiple DUM messages, only
the last DUM message contains the correct sizep value. I am
concerned some implementations will not be prepared for this case.
Sizep should probably not include HTTP headers in a standard HTTP
profile.
Putting all this together, I feel that using the sizep parameter for
Content-Length indication does not really work.
I agree.
Shouldn't we better introduce a Content-Length named parameter to
AMS messages that does what we really want: Promise a length that
can be used for HTTP Content-Length header correction?
Does sizep have any useful purpose at the lower (application data)
level it is being used right now? Should we delete sizep from the Core
and add AM-CLP parameter instead (application message content length
prediction)?
Alex.