In OCP the callout server does not need to return any data, if the
processor preservers all data and sends "Kept" parameters with its
DUM messages. During the data trickling phase, the callout server
returns DUY messages with small sizes and after scanning is complete
it sends one final DUY message that allows the complete rest to be
sent to the client. The 5MB is only transferred one way, and only a
few bytes for the DUY messages is sent back.
This works in theory. The hard part is to convince the processor to
preserve/keep 5MB of data. It is a lot of overhead for the processor
to keep all the data, and I am sure that a general-purpose
implementation may not be able to keep the whole thing by default.
I know at least one ICAP/1.0 implementation that can keep copies of very large
files (swapping to disk if needed).
Virus filtering is one of the most wanted callout service today. It makes sense
to design OPES processors in a way to support those best.
While many OPES processors are caching proxies and because virus filter
services often do not change data, it makes sense to store already the original
data into the cache, being prepared to replace it later if required and so
preserving all data anytime.
OCP does not (yet?) have any mechanisms to (a) encourage preservation
or to (b) notify the server that the preservation buffer is limited to
X bytes and any spills will not be preserved. Do you think it should
be left to external negotiation channels (configuration files, etc.)?
Or should we try to support this directly? Which mechanism?
I do not think right now that exchanging this data or even negotiating on this
will really change something.
A processor may or may not have limitation for its data preservation
capabilities. Preferences of the callout server will probably not change
Callout servers may like to make intensive use from preserved data but be
definition need to be prepared that the processor stops to preserve more data;
it has to handle that anyway. No need for a special announcement I guess; it
won't change the algorithm I guess.
Am I missing something?
Do you know of examples where exhanging of that information will make a