Here's an interesting item

hih_tv-Greg
hih_tv-Greg
Joined: 11 Feb 05
Posts: 94
Credit: 31815
RAC: 0
Topic 190101

Hello all,
Hope everyone is crunching away...anyway. Here is something I found while fiddling with my machine.
This is the specs with Windows 98 installed:
Machine on Windows98

10/31/2005 06:09:49 PM||Running CPU benchmarks

10/31/2005 06:10:46 PM||Benchmark results:

10/31/2005 06:10:46 PM||Number of CPUs: 1

10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU

10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU

10/31/2005 06:10:46 PM||Finished CPU benchmarks

And, this is the results with Windows XP installed:
Machine on WindowsXP

10/30/2005 02:15:31 PM||Running CPU benchmarks

10/30/2005 02:16:30 PM||Benchmark results:

10/30/2005 02:16:30 PM||Number of CPUs: 2

10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU

10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU

10/30/2005 02:16:30 PM||Finished CPU benchmarks

One and the same machine just the OS is different.
Hypertheading was active in both configuations.

Greg

Walt Gribben
Walt Gribben
Joined: 20 Feb 05
Posts: 219
Credit: 1645393
RAC: 0

Here's an interesting item

How do the elapsed and CPU times compare?

Hyperthreading won't do you any good on Win98, it only supports one processor. So that one processor gets full use of the CPU.

Hyperthreading does mess up benchmarking - it thinks there are two full processors, but you really just have one. If you have a lot of spare time, you might try it with hypertheading turned off, and again with hyperthreading turned back on again but with your preferences set to use just one processor.

Walt

Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5385205
RAC: 0

You will likely see a

You will likely see a "spread" of values ... I ran the benchmarks 10 times before I was confident I had a good average of the "actual" value.

Walt Gribben
Walt Gribben
Joined: 20 Feb 05
Posts: 219
Credit: 1645393
RAC: 0

RE: You will likely see a

Message 18928 in response to message 18927

Quote:
You will likely see a "spread" of values ... I ran the benchmarks 10 times before I was confident I had a good average of the "actual" value.

How do you keep the average consistent? Like what happens five days later when BOINC decides the old numbers aren't good enough?

Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5385205
RAC: 0

RE: RE: You will likely

Message 18929 in response to message 18928

Quote:
Quote:
You will likely see a "spread" of values ... I ran the benchmarks 10 times before I was confident I had a good average of the "actual" value.

How do you keep the average consistent? Like what happens five days later when BOINC decides the old numbers aren't good enough?


Run it till you get numbers you like ... or edit the xml files ...

Fundamentally the use of Dhrystone and Whetstone are only the "best" of the worst choices. In other words, there is not much else, but, when you read things like:

"Q: Why does the rating vary between sequential runs?
A: On most systems, the value of the rating shouldn't change by more than about +/-5% from run to run. On systems with limited memory it may vary by +/-10% due to memory swapping. If you're seeing variations higher than this, some hardware or software is probably to blame. Do note that "limited memory" depends on operating system, installed drivers, running programs, etc. A badly configured system with 64MB may be worse memory-wise than an 16MB system..."

SANDRA Pro

(I was just looking at some other pages about Whetstone ...)

But it is not that I have ever been of any other opinion ... I never thought that the "synthetic" benchmarks were good for much other than to create a number. All in all, we are still avoiding the problem. You either have to directly count the actual ops, or you need to bench with the actual work. PERIOD (Paul's opinion)

I don't like counting the ops, because that does add overhead, and even 1% is a lot ... and to do accurate accounting, WHICH I ALSO THINK IS NECESSARY, the probability is that the overhead will be at least that much, if not more. The real problem is that virtually none of the applications we are running are deterministic.

That is why my proposal is as elaborate as it is. It is actually not that elaborate at all, ONCE YOU GET THE DRIFT OF IT. We run some work with instrumentation and "eat" the cost. But, the work is real work, not fake, and the participant gets full credit for all the time spent, including the accounting. But, then we don't put the accounting burden on most of the work performed.

That is it in a nutshell.

I do not believe that the latest change/refinement Dr. Anderson just added will not materially affect what is going on. All it will do is improve the "balance" of the FP/Integer ratios and it may reduce the wide variances we see in credit claims. But, I do not expect that ... in theory, we should be able to see an almost immediate change if he has already adjusted the SETI@Home numbers. Within a week or two we can start to "spot check" work to see if it has come down in "enthusiasm" ...

I have been looking at the code and I may take the 90,000 some odd results I have in my data base and play with his tool to see what it does. I have also been trying to think of a way that may allow us to use something of the sort to reduce the spread. But, till we have more data, and I "test drive" his tool ... no idea ...

Though, since I did stop doing SETI@Home work a week or so ago, I may also be a good test case. I can also update my data collection tools and start to do SETI work and also start to capture the new data ...

===== Edit
One of my biggest objections to the current system is that it can be "hacked" and since this was one of the issues in Classic, and there is no protection against subtle "hacks" to inflate a participant's score ... well ... sorry ...

Credit is a validation of what I have contributed, any allowance of compromise cheapens my contribution ...

hih_tv-Greg
hih_tv-Greg
Joined: 11 Feb 05
Posts: 94
Credit: 31815
RAC: 0

Hay all, I'm going to shut

Hay all,
I'm going to shut down now and deactivate the Hypertheading...will post the results as soon as the system is up.

Greg

hih_tv-Greg
hih_tv-Greg
Joined: 11 Feb 05
Posts: 94
Credit: 31815
RAC: 0

Well here is the results

Well here is the results without Hypertheading:

11/01/2005 07:59:36 PM||Running CPU benchmarks
11/01/2005 08:00:33 PM||Benchmark results:
11/01/2005 08:00:33 PM||Number of CPUs: 1
11/01/2005 08:00:33 PM||1478 double precision MIPS (Whetstone) per CPU
11/01/2005 08:00:33 PM||2294 integer MIPS (Dhrystone) per CPU
11/01/2005 08:00:33 PM||Finished CPU benchmarks

Still running Windows 98 OSR-2, here are the old numbers just for referance:

"Machine on Windows98

10/31/2005 06:09:49 PM||Running CPU benchmarks
10/31/2005 06:10:46 PM||Benchmark results:
10/31/2005 06:10:46 PM||Number of CPUs: 1
10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU
10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU
10/31/2005 06:10:46 PM||Finished CPU benchmarks

Machine on WindowsXP
10/30/2005 02:15:31 PM||Running CPU benchmarks
10/30/2005 02:16:30 PM||Benchmark results:
10/30/2005 02:16:30 PM||Number of CPUs: 2
10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU
10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU
10/30/2005 02:16:30 PM||Finished CPU benchmarks"

Greg

hih_tv-Greg
hih_tv-Greg
Joined: 11 Feb 05
Posts: 94
Credit: 31815
RAC: 0

O.K. here is another item...I

O.K. here is another item...I just installed 512meg's of ram into this machine, upon boot up the bois reports 504megs of ram, (with the 256meg I had the bios would report 254megs of ram).

Without reguard of the above events I ran a benchmark off of Boinc to see if there would be any speed gained with the added ram. here is the results:

Machine on Windows98/Multithreading off, 512Meg Ram, Boinc 4.45
11/05/2005 06:09:29 PM||Running CPU benchmarks
11/05/2005 06:10:26 PM||Benchmark results:
11/05/2005 06:10:26 PM||Number of CPUs: 1
11/05/2005 06:10:26 PM||1468 double precision MIPS (Whetstone) per CPU
11/05/2005 06:10:26 PM||2190 integer MIPS (Dhrystone) per CPU
11/05/2005 06:10:26 PM||Finished CPU benchmarks

Here are the old values that started this threaded, just for background:

Machine on Windows98/Multithreading off, 256Meg Ram, Boinc 4.45
11/01/2005 07:59:36 PM||Running CPU benchmarks
11/01/2005 08:00:33 PM||Benchmark results:
11/01/2005 08:00:33 PM||Number of CPUs: 1
11/01/2005 08:00:33 PM||1478 double precision MIPS (Whetstone) per CPU
11/01/2005 08:00:33 PM||2294 integer MIPS (Dhrystone) per CPU

Machine on Windows98/Multithreading on, 256Meg Ram, Boinc 4.45
10/31/2005 06:09:49 PM||Running CPU benchmarks
10/31/2005 06:10:46 PM||Benchmark results:
10/31/2005 06:10:46 PM||Number of CPUs: 1
10/31/2005 06:10:46 PM||1379 double precision MIPS (Whetstone) per CPU
10/31/2005 06:10:46 PM||2242 integer MIPS (Dhrystone) per CPU
10/31/2005 06:10:46 PM||Finished CPU benchmarks

By the way the output below is the machine running the new Boinc 5.2.6
Machine on WindowsXP/Multithreading on, 256Meg Ram
10/30/2005 02:15:31 PM||Running CPU benchmarks
10/30/2005 02:16:30 PM||Benchmark results:
10/30/2005 02:16:30 PM||Number of CPUs: 2
10/30/2005 02:16:30 PM||1303 double precision MIPS (Whetstone) per CPU
10/30/2005 02:16:30 PM||1599 integer MIPS (Dhrystone) per CPU
10/30/2005 02:16:30 PM||Finished CPU benchmarks

The specific of the hardware of this machine is in my sig. Can anyone explain what is going on?

Greg

Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5385205
RAC: 0

Yes ... :) The benchmarks

Yes ... :)

The benchmarks are unstable ...

If you run them 10 times in a row, there is a very good chance that you will see this sort of variability. This is probably the second most important indictment of the benchmarks. The first of course being that they do not characterize the systems very well ...

hih_tv-Greg
hih_tv-Greg
Joined: 11 Feb 05
Posts: 94
Credit: 31815
RAC: 0

Hi Paul, Ahhh to bad, just

Hi Paul,
Ahhh to bad, just wanted to know if I was making some headway on inprovments to the system.

Greg

Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5385205
RAC: 0

Greg, The "best" way is to

Greg,

The "best" way is to actually do a running average over some period of time using a number of actual work units.

In other words, take the completion time of, say, 10 work units; average that time. Make the change, collect time on another 10 work units, average the times, compare the averages ...

Though I keep saying it, there are plenty of people that still look at the benchmark numbers as if they have meaning ... basically we proved otherwise back in the Beta test. It is just that there has not been a "good" idea of what to do to replace the current system (well, I have one now, but it is not likely to see daylight anytime soon).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.