If we are to support copying (per data chunk or per message), we need
to decide when the copying commitment expires. In other words, when
the OPES dispatcher can release data previously sent with the "copied"
flag set.
If the callout server uses "data-have-as-is" message that matches the
entire data block sent by the dispatcher, then the dispatcher can
probably free the data (we do not have to support duplication of data,
I guess; if a byte range has been sent, it is unlikely to be sent
again).
However, what happens if there is a partial match? Should the
dispatcher keep the leftovers until the end of the message? Until
copied data with higher offset is reused? Until wont-need message is
received? A combination of these?
There is also a potential deadlock situation here. If the amount of
dispatcher buffer is limited, the dispatcher may stop sending more
data to the callout server, and the callout server may need more data
to respond (and let OPES dispatcher clear some of the copied chunks).
We may want to address this explicitly instead of relying on
timeouts.
Copying data introduces state and blocking (due to buffer
limitations). We need to find a simple way to support copying in many
environments while keeping the protocol simple enough. I am seeking
suggestions on what to support or where to draw a line. My current
intention is to support "getting out of the loop" for OPES processor
and nothing else, to optimize for the most common case. Comments?
Thanks,
Alex.