Here's an interesting item

Joined: 11 Feb 05
Posts: 94
Credit: 31,815
RAC: 0

RE: Greg, The "best" way

Message 18936 in response to message 18935



The "best" way is to actually do a running average over some period of time using a number of actual work units.

In other words, take the completion time of, say, 10 work units; average that time. Make the change, collect time on another 10 work units, average the times, compare the averages ...

Though I keep saying it, there are plenty of people that still look at the benchmark numbers as if they have meaning ... basically we proved otherwise back in the Beta test. It is just that there has not been a "good" idea of what to do to replace the current system (well, I have one now, but it is not likely to see daylight anytime soon).

Hey Chief

Good to hearfrom you...I'll do that since reading some of the information from your links, raised some quastions concerning benchmark(ing).


Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5,385,205
RAC: 0

look at the performance

look at the performance lecture which discusses some of the more common benchmarks ...

Probably the best is the "Spec-something" where you have to really define what your test regime was. Which of course is another problem ... because you can still "cook" the results, but you have to document the recipe ...

Oh, and the numbers are useless except when you compare identical configurations with identical configurations ... :)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.