Sounds good to me.
From: Volker Hilt [mailto:vhilt(_at_)dnrc(_dot_)bell-labs(_dot_)com]
Sent: Tuesday, July 24, 2001 6:31 AM
To: Rahman, Rezaur; ietf-openproxy(_at_)imc(_dot_)org; Frank Berzau
Subject: Re: Protocol Performance
I haven't heard of anyone doing any performance analysis
between SOAP and
ICAP. I think your work will help openproxy group to choose
protocol. I would suggest you post the performance
you are planning on to this mailing list, so that the group
can comment on
its applicability to the OPES framework.
Our main goal is to measure the following two aspects of both
- additional bandwidth introduced by the protocol
(compared to the original message size)
- CPU cycles needed for protocol processing (e.g. for
* the creation of headers,
* message marshaling,
* transferring data to/from network,
These aspects seem to be most significant during the run-time of a
remote call-out protocol, in particular if a large number of requests
has to be processed. (We're not planning to look into non-run-time
aspects such as design issues, ease of implementation, etc.)
The evaluation-setup we're currently thinking of are
on existing protocol implementations. We will generate fixed
HTTP-Requests and Responses that we will feed into the client-side
implementation of the protocol stacks. We will not include the message
parser and the rule processor in the performance measurements. Instead
we planning to feed the prepared messages directly into the
the protocol itself. This excludes all non-protocol related
the measurements (even if they would be the same for both
the server side we're planning to use a dummy service, that simply
returns the message it has received from the client.
The measurements itself will be taken by introducing checkpoints into
the protocol code. This should enable us to measure bandwidth
as well as
required CPU-cycles on a fine-grained level.
An implication of the performance evaluation using existing
implementations is that the quality of the implementation itself has
impact on the result of the analysis. To minimize this effect, we are
trying to pick two implementations that are somehow
comparable. But even
if the implementations differ, the analysis will still
measures of the fractions of time that are spent on header processing,
message marshaling and sending/receiving data.
I would also like to see a comparison between a caching box
running with and without ICAP, or with or without SOAP (if any
exists yet). Kind of an ICAP bake-off. If we had a ICAP server
which would not do any real processing, but just respond back
to the server, plus a load balanced setup so we don't see the
server become the bottleneck. That should give us an idea about
the performance impact when enabling ICAP.
Would that be in the scope of what you're planning?
Currently we're not planning to do such measurements. However, our
analysis will show the costs of using a call-out protocol
which seems to
be an important component needed to determine the brake-off point upon
which the use of a remote callout server is preferable to