[S5R3/R4] How to check Performance when Testing new Apps

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686042601
RAC: 594764

I guess one of the remaining

I guess one of the remaining mysteries is the connection between the

- frequency displayed in the task name (f_t),
- the frequency in the comamnd line (f_c)
- and the frequency of the sky grid file (f_g).

As far as I can see ,

f_g = ceil(f_c/10.0)*10.0

but the connection between f_t and f_c is less obvious. It's close to

f_c ~ f_t + 0.15

but not exactly that.

CU
Bikeman

archae86
archae86
Joined: 6 Dec 05
Posts: 3145
Credit: 7023184931
RAC: 1829158

RE: - frequency displayed

Message 77825 in response to message 77824

Quote:
- frequency displayed in the task name (f_t),
- the frequency in the comamnd line (f_c)
- and the frequency of the sky grid file (f_g).


To Bikeman's list of frequencies for a work unit, I'd add the numbers shown in col 1 of the 10000-line result zipped result file.

For example, for a result file named:
h1_0905.20_S5R3__423_S5R3b_0_0

Col 1 contains values ranging from:
905.3934422 to
905.4095811

Assuming 2303 distinct, with a nearly uniform step of 0.0000067105 or 0.0000067106 between distinct value, though a few cases of double and still fewer of triple that.

It is tempting to suppose that these are the "real" frequencies for which computation was done for the Work Unit, in which case the range of frequencies would span from about .1934 above that implied by the task name to .2096 above for the specific case cited.

That amount is not constant across all the results I've looked at, though so far they seem pretty similar to this.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921893
RAC: 16834

RE: I guess one of the

Message 77826 in response to message 77824

Quote:

I guess one of the remaining mysteries is the connection between the

- frequency displayed in the task name (f_t),
- the frequency in the comamnd line (f_c)
- and the frequency of the sky grid file (f_g).

As far as I can see ,

f_g = ceil(f_c/10.0)*10.0

but the connection between f_t and f_c is less obvious. It's close to

f_c ~ f_t + 0.15

but not exactly that.


I haven't got the code in front of me, so that's also from the top of my head (and actually Reinhard wrote the Workunit generator, not me):

- f_c is the actual base frequency where the search for this task starts.
- f_g is the sky-grid file for this frequency, your formula is correct.
- f_t is actually part of another name: all task names begin with the name of the "lowest" data file that is required for this task.

We need some "wings" below the base frequency and above the top frequency of the task, the latter being f_c + bw (bandwidth, I think the command-line option is --FreqBand). The size of the necessary "wings" also increases with frequency, you'll need more data files with higher frequency even if the bandwidth stays the same; so also f_t - f_c will increase with f_c.

The first column in the result file should always be between f_c and f_c+bw. The length of the output file is limited, so we keep only the n "most interesting" results (IIRC n=15000). Of course they are not necessarily equally spaced in frequency.

BM

BM

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686042601
RAC: 594764

Ah, that explains it,

Ah, that explains it, thanks!

As to the idea of a reference unit that would perform an all-sky search with a coarse grid in standalone mode, I think this is a very interesting idea.

This could be used for:

-Benchmarking new app-versions
-Benchmarking computers
-As an extreme overclocking-test tool (run until it fails, but avoid submitting error results as the test would run stand-alone and not under BOINC)
-A tool to reproduce errors encountered on a particular platform ...)

The only problem I see currently that would prevent the immediate construction of such a reference test suite is that I know of no easy way to determine the CPU seconds used by a process under Windows. Under Linux and other UNIX like OSes you would just use time, but there is nothing comparable under windows, right?

Maybe a future version of the E@H app should log this itself in stderr.txt before completing the task in standalone mode?

CU

Bikeman

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: I just found this in

Message 77828 in response to message 77822

Quote:
I just found this in another thread and think it might be suitable to be answered here ..... BM


Thank you, Bernd. :-)
I've just had a long read of this terrific post .... and I think I understood most of it! :-)
Congrats to the guys for deducing the 10Hz thingy too ....
I'd like to entertain the idea of a test suite ( mainly for the hell of it ), but it will clearly need more thought.
I do recall there is a ( somewhat arcane ) way to record/summate time used per process ( thread really ) under Windows but it escapes me at the moment. It's do-able, but not self evident - I think to be ultra-precise it requires a hack of an application program's executable though. Under Windows NT at least there are/were performance monitor counters available through system calls/callbacks, but in practice you could only sensibly apply it to code fragments .... more recently I think there is 'GetProcessTimes' - I could be spouting rubbish though .... :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

[url=http://msdn2.microsoft.c

[url=http://msdn2.microsoft.com/en-us/library/ms683223(VS.85).aspx]GetProcessTimes Function[/url]

Says it's good for 2K forward, probably NT4 as well.

Alinator

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686042601
RAC: 594764

Hi! I guess there is

Hi!

I guess there is already CPU time stats code in the app (to report it to the core client) but I'm not sure that information is collected when the app runs in standalone more (w/o BOINC), and it surely isn't printed.

What about all those other benchmarks like prime95 etc: do they report real time or CPU time?

CU
Bikeman

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: [url=http://msdn2.micro

Message 77831 in response to message 77829

Quote:

[url=http://msdn2.microsoft.com/en-us/library/ms683223(VS.85).aspx]GetProcessTimes Function[/url]

Says it's good for 2K forward, probably NT4 as well.

Alinator


Yup, that's the one! :-)

The deal with pre-emptive systems ( Windows 3.x was 'co-operative' multitasking ) is that the threads in the outer/lower privilege rings ( non-kernel ie. users ) can't by themselves determine their own timings - simply because they may be interrupted at any moment ( stop/start/sleep/switch .... ). They can refer to system timers to get elapsed ( real ) times from some earlier point, like seconds since system startup, but this doesn't yield summated time spent within a thread. Only a ring zero function can accurately do that by knowing the time per slice as allocated by the task scheduler, adding all the slices up for you, and then handing/serving up a summary via GetProcessTimes say.

One can estimate the actual total time spent by a CPU running one's code by multiplying real-time ( wall clock if you like ) by CPU usage ( a fraction often quoted as a percent ) but you're assuming other system behaviours now. Windows Task Manager ( Processes pane ) will give you this, or even fuller reporting with Process Explorer say : my dual core has two einstein_S5R3_XXXX 's at present, both @ 48 - 50 % each when unmolested, but that's the rub - it's only reliable for long times when undisturbed for those long times. Avoiding this by setting a thread priority to realtime is never a good idea - dependencies within the operating system will scupper this [ and your system ].

Quote:
[aside]Another aspect I can get annoyed about, if it wasn't so useful like my Trend Micro anti-virus, is the system tray. Most programs I have there, the modern equivalent of the old DOS terminate-and-stay-resident crew, can/will wake up periodically and muck about somewhat - including establishing Internet connections, downloads, scans and all that they may entail. Apple/I-Tunes would be the worst abusers/offenders of this facility in my view....[/aside]

Cheers, Mike

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: Yup, that's the one!

Message 77832 in response to message 77831

Quote:


Yup, that's the one! :-)

The deal with pre-emptive systems ...

Quote:
[aside]Another aspect I can get annoyed about, if it wasn't so useful like my Trend Micro anti-virus, is the system tray. Most programs I have there, the modern equivalent of the old DOS terminate-and-stay-resident crew, can/will wake up periodically and muck about somewhat - including establishing Internet connections, downloads, scans and all that they may entail. Apple/I-Tunes would be the worst abusers/offenders of this facility in my view....[/aside]

Cheers, Mike

No argument there on the first part. IIRC this is the problem in 9x with CPU time. MS just didn't build the necessary functionality into kernel32.dll by default, but there were third party workarounds for that.

Regarding your pet peeve topic, it isn't the System Tray I mind so much as the tendency for every software publisher to assume their program is so important it should be there by default.

I mean if I don't want your icon cluttering up my desktop, it is sheer hubris to assume I want it taking valuable space on the Taskbar (even if it is smaller)!

News Flash to developers.... That's what the Start Menu is for. If I bother to install your stuff, you can rest assured I'm not going to forget it's there, and I'm fully capable of deciding when I want it to run, thank you very much! ;-)

How could I have forgotten this part.... A relatively new trend now is to make it so you have to hack the registry to get rid of it from the tray as well. For me, this almost guarantees your application a quick trip to the Recycle Bin.

Wow... I haven't had a Dvorak style rant in a while.... It felt good! :-D

Alinator

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: No argument there on

Message 77833 in response to message 77832

Quote:
No argument there on the first part. IIRC this is the problem in 9x with CPU time. MS just didn't build the necessary functionality into kernel32.dll by default, but there were third party workarounds for that.


The 'co-operative' expectation of Windows 3.x quickly became an utter joke too - the MS recommendation at the time was for third party applications to politely release/yield their own thread to others - like that ever happened, much!! :-)

Quote:
[aside]
Quote:
Regarding your pet peeve topic, it isn't the System Tray I mind so much as the tendency for every software publisher to assume their program is so important it should be there by default. ...Wow... I haven't had a Dvorak style rant in a while.... It felt good! :-D

Ab - so - bl**dy - lutely! :-)

Adobe, Macromedia Flash and a raft of others have hit the R-Bin pronto for exactly that. I don't just mean the System Tray component of the product - I punt their entire product down the Happy Black Hole. I've have yet to see a single installer ( please quote back at me any exceptions ) that gives System Tray installation as a distinct/prominent/separate option. I can and do live happily without visiting some web page that 'requires' Flash or any other BrandX for it's use. Like companies that don't have real people answering their phones [ anyone for a 'press 6 to flush our stock price down the toilet' option? :-) ] I simply cancel my business with them - no fuss, no notice, no complaint, no radar reflection - just no more money. :-)

I think there are alot of deluded next-quarterly-report-hugging managements out there that think that : the money they are saving by setting up mechanisms to avoid paying people to speak with their customers will satisfy their shareholders in the long term. [ Free Handy Hint #1 - you have to maintain financial solvency to achieve that. Free Handy Hint #2 - you need the customers' money coming in to nail #1 .... ]
[/aside]

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.