RE: OPES protocol, pre-draft

Let me propose a simplification of the protocol: remove 
[copied] flag and make this behavior compulsory.

Reasons:

1. The main goal of chunks is to reduce network traffic. OPES server
implementation that discards message chunk by chunk as soon as one 
is send to callout server does not look a good idea. Even if call 
starts before the complete message is received by the OPES processor 
it should be able to assemble the complete message anyway. In 
asynchronous system OPES processor can not rely on ability of 
the callout processor to keep pace with the speed of incoming traffic.

2.Reliability. If the original message is discarded by OPES processor 
before transaction with the callout processor is completed callout 
processor failure (including connection outage) forces OPES processor 
to fail on this message, while preserved copy provides multiple 
ways for recovery - default action on callout server unavailable, 
repeat request to a different callout server. And the message 
was already there, savings from premature discarding are very 
small.

3. catch 22: if message processing is simple and anticipated 
processing time is small - so are the savings from message discarding.
If anticipated processing time is significant - savings may be 
bigger but so are the risks related to failure and needs for 
reliable recovery. 

Same thing with the message size - for small messages savings are 
negligible, for large messages savings from ability to continue 
without returning message body (e.g. in virus checking) out weight 
savings from message discarding.

4. For streaming data policy may be different but still predefined: 
for live streaming data should me discarded (or alternative action 
taken) if delay is too long, for preload situation is the same as 
with large message.

All of this shows that benefits of controlled discarding are too small 
to justify complications in the protocol and system implementation.

My .10

Oskar
   
    

> -----Original Message-----
> From: owner-ietf-openproxy(_at_)mail(_dot_)imc(_dot_)org
> [mailto:owner-ietf-openproxy(_at_)mail(_dot_)imc(_dot_)org]On Behalf Of Alex 
> Rousskov
> Sent: Monday, March 17, 2003 7:49 AM
> To: ietf-openproxy(_at_)imc(_dot_)org
> Subject: Re: OPES protocol, pre-draft
>
>
>
> Markus,
>
>       We probably need a "sticky" [copied] flag meaning "I am going
> to have a copy of every byte from now on, until further notice". This
> way the callout server can remove itself from the loop earlier.
>
> Thanks,
>
> Alex.
>
>
> On Sun, 16 Mar 2003, Markus Hofmann wrote:
>
> > Please apologize my late reply, but this just came to mind when
> > catching-up with all the emails after coming back from vacation and
> > reading through them for a second time on the plane...
> >
> >
> > Martin Stecher wrote:
> > > Example 2:
> > > Callout server analyzes beginning of application message and detects
> > > that it is not interested in this app. message:
> > >
> > > A. Traditional approach, ICAP/1.0 like with multiple preview
> > >     0. C: Start of app. message 1
> > >     1. C: Here is preview chunk 1
> > >     2.                                S: Need another preview chunk
> > >     3. C: Here is preview chunk 2
> > >     4.                                S: Not interested (204 resp.)
> > >     5. C: Start of app. message 2
> > >
> > > B. Proposed OPES protocol
> > >     0. C: Start of app. message 1
> > >     1. C: Chunk 1 of data             S: Start of app. message 1
> > >     2. C: Chunk 2 of data
> > >     3. C: Chunk 3 of data             S: Not interested
> > >     4. C: Chunk 4 of data
> > >     5. C: Start of app. message 2
> > >
> > > In this case B sent more data and data chunks 3 and 4 will be ignored
> > > by the callout server.
> > This might be obvious, but please allow me to add the following for
> > clarification only... Chunks 3 and 4 can only be ignored (i.e.
> > discarded) by the callout server if the OPES processor has sent them
> > with the [copied] flag set in the respective <data-have amid offset
> > size [copied]> message(s). If the [copied] flag was not set, the
> > callout server must not ignore chunk 3 and 4, but simply copy/send
> > them back to the OPES processor. Obviously, this can be done without
> > further processing (i.e. servicing), but they still need to be sent
> > back, since the OPES processor might no longer have a copy of these
> > chunks. Note, that this is true even if chunks 1 and 2 were sent with
> > the [copied] flag set.
> >
> > Thanks,
> >    Markus