On Fri, 28 Feb 2003, Martin Stecher wrote:
For example, consider this situation: Callout server processing of
each response chunk is long and "expensive". The callout server is
processing the first chunk, and 9 next chunks are still in its TCP
buffers waiting to be parsed and processed. The user aborts its
request. There is no reason for the callout server to continue, but it
will not know until it gets a <xaction-end> message. Without the
optimization in question, the callout server will not know until all
10 chunks are processed.
Wait a second, to avoid a misunderstanding. The example you are
describing is this really using a <xaction-end> message to abort? I
thought it should actually be a <producer-end amid reason> message
instead of <xaction-end xid reason> because other application
messages within this transaction are not affected, are they?
You may be right, depending on the application protocol and state. For
example, it makes no sense to continue processing an _uncachable_ HTTP
response if the client has left and the whole transaction (the HTTP
request-response pair) can be terminated. If response is cachable, it
may make sense to continue, especially if a large portion of the
response has been received already, and if the cache stores
post-processed responses.
I doubt OPES can require usage of a particular -end message because
optimal behavior would depend on application protocol and state. We
can only recommend that the implementations on both sides of the
protocol leave as much decision freedom for the other side as
possible; if it makes sense to send a message-abort instead of a
transaction-abort, the former message should be sent.
Here is a related question: should we let the other side affect the
abort decision on OPES level? Perhaps the callout server is doing some
logging or accounting and MUST see every byte received by the OPES
dispatcher, even if a part of the transaction is aborted by the
dispatcher. Should we add some kind of <xaction-need-all> message? Or
should we assume that the dispatcher always knows callout server needs
and vice versa?
I am somehow undecided between solutions 0) and 2).
Maybe it makes sense to go for 2) and make clear in the protocol
that the abort messages MUST be sent via the normal channel.
It could then be an option or even extension that is not handled
in the protocol from the beginning to send these messages in
copy on a fast channel.
Yes, that is how (2) should be documented/implemented. It is an
optional (MAY) optimization feature for the sender. I think it MUST be
supported by the recipient though to make optimization decisions
simpler.
In general, I lean towards allowing out-of-sent-order messages because
without them the protocol is too slow/inefficient for handling large
transfers. I think we ought to allow more interactivity between
callout server and the dispatcher because such interactivity may be
the only way to implement certain adaptation-specific features. We just
need to make this optimization simple and safe (and solution (2) may
achieve that).
Here is a motivational example. Imagine and OPES system that modifies
certain rare URLs (links) in HTML responses based on a large set of
rules. The rules tell what URLs should be rewritten and how. Maybe
this is a CDN or another kind of a site mirror, does not matter. The
rules are maintained at the OPES dispatcher side (for reasons outside
of our control). The rule set is too large to be transferred to the
callout server. Thus, when the callout server detects a candidate URL,
it has to ask the dispatcher whether this URL needs modifications and
what modifications it needs. This can probably be done via some OPES
extensions. If these request/response pairs are sent via normal "data"
channels, the processing may be very slow and require a lot of
outstanding state. If the request/response pairs can go on a separate
connection, the problem goes away; the delay to process each rare URL
does not depend on the length of the message (or the total length of
outstanding chunks, to be precise).
The question is, of course, whether we should care about such
situations or declare them out of OPES scope instead. We can say, for
example, that if an OPES implementation needs interactive dialog
between OPES agents, that dialog should be supported on separate
non-OPES channels. I am not sure this would be the right solution
because the dialog will probably be related to OPES communications
and, hence, can benefit from being within the OPES framework.
Does anybody have good examples where interactivity among OPES agents
would be required and would benefit from being performed over OPES
channels?
Thank you,
Alex.