"Andrew" == Andrew Welch
<andrew(_dot_)j(_dot_)welch(_at_)gmail(_dot_)com> writes:
Andrew> On 21/12/2007, Colin Paul Adams
<colin(_at_)colina(_dot_)demon(_dot_)co(_dot_)uk> wrote:
>> >>>>> "Andrew" == Andrew Welch
<andrew(_dot_)j(_dot_)welch(_at_)gmail(_dot_)com>
>> writes:
>>
Andrew> On 21/12/2007, Colin Paul Adams
<colin(_at_)colina(_dot_)demon(_dot_)co(_dot_)uk> wrote:
The cost of the JVM start-up cannot be eliminated, but it can
>> >> be amortised. For a fair comparison, you need to know what
>> this >> overhead is, so you can make sensible judgements.
>>
Andrew> I don't think it needs to be amortised... If you start the
Andrew> timing at the correct point and ignore the first few runs
Andrew> then I'd say you can safely forget about JVM startup and
Andrew> warm-up for the remaining runs.
>>
>> Then you've ignored it completely, so you don't get a fair
>> comparison.
Andrew> aahhh I see what you mean.... (the penny drops :)
Andrew> I don't agree though - if you are comparing say Gestalt
Andrew> Win32 version with Saxon running on Windows, then surely
Andrew> it's only really fair to make the comparison after the
Andrew> Java bytecode has become native machine code - after JVM
Andrew> startup.
That depends upon what you mean by fair (that is, what are you trying
to measure?).
So, if what you are interested in is the performance of a web-server
system, then yes, that is probably the only meaningful measurement.
And for Gestalt, you would have to adopt the same methodology, in a
Goanna servlet (although i doubt if it would be significant, but I
have never measured it, so i could be very wrong).
If you workload (which is typically what I use XSLT for, but I am very
probably highly atypical, as I don't use XSLT much at all these days,
except for testing) is occasionally transforming gobodoc xml into an
html page, then ignoring the JVM start-up time would give me an absurd
conclusion, as it completely dominates the overall run-time.
In general, the run-time of an XSLT processor is going depend upon the
sum of a number of factors. For a Java-based processor, the JVM
warm-up time is such a factor (it is some kind of non-constant
workload-dependent function).
So to fully characterize the performance, you are going to need to specify a
number of parameters. In general, I mean. For a given workload,
processor comparisons are generally much simpler to do.
The only real value of a general inter-processor benchmark has, I
think, is to give a prospective user an idea of which processors to
measure against their own workloads. But still, I find it an
interesting topic (for about 9 years my principle job was a
performance analyst on mainframe systems).
--
Colin Adams
Preston Lancashire
--~------------------------------------------------------------------
XSL-List info and archive: http://www.mulberrytech.com/xsl/xsl-list
To unsubscribe, go to: http://lists.mulberrytech.com/xsl-list/
or e-mail: <mailto:xsl-list-unsubscribe(_at_)lists(_dot_)mulberrytech(_dot_)com>
--~--