[S5R3/R4] How to check Performance when Testing new Apps

archae86
archae86
Joined: 6 Dec 05
Posts: 2,930
Credit: 3,715,017,990
RAC: 4,727,009

RE: RE: ( edit ) Here's

Message 77794 in response to message 77792

Quote:
Quote:

( edit ) Here's an example ( thanks to archae86 ) of where the analysis wobbles off, by the points straying from ~ sinusoidal pattern :

....

What happened there? You may well ask .... :-)

I haven't plotted the points but I can see exactly what you are referring to :).

Something happened probably during or around seq# 40 which gave a step improvement of close to 10% in crunching performance. Could it have been an app change to a faster app or could it have been hardware improvements like upping the overclock a bit :). Maybe Peter might have some thoughts on this.


It appears to be the version 4.07 to 4.11 upgrade. I no longer recall whether that was seen as a speed upgrade at the time or not, but the files show that the very next results after the ones Mike shows are labelled as 4.11, and these as 4.07.

I maintain a bit of a queue, and the data I sent to Mike are collected by BoincLogX, which uses the "branding" at download time, not the more sophisticated scanning of stderr out used by methods posted here recently.

So, in my raw data, not only are "mixed" results right at the conversion boundary mislabelled, but also any results already downloaded at conversion time.

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 1,960
Credit: 334,318,949
RAC: 825,707

As noted in the "Power Users"

As noted in the "Power Users" thread (Windows), I'm dedicating host 1001562 to a timing run with 4.32 - all tasks at 0909.15Hz will be crunched exclusively with 4.32: host is a Q6600 at stock 2.4GHz, other projects will be suspended for the duration. I'm starting at sequence number 502 - let's see how far this one will go! Feel free to use this as a data source for Gary's script - I'll have to do the data collection manually, not having a Linux box to hand. Just got three sub-800 Einsteins to finish off first, and some SETI, but we should be in full flow by the morning.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,354
Credit: 48,608,572,303
RAC: 57,715,993

RE: I'll have to do the

Message 77796 in response to message 77795

Quote:
I'll have to do the data collection manually, not having a Linux box to hand.

If you like, I'll collect it for you and email it to you. Just as an example, this is what the data for that machine currently looks like on the website:-

0777.50,326,24725,mixed
0777.50,004,33027,mixed
0777.50,008,35535,4.26
0777.50,010,35235,4.26
0777.50,013,34562,4.26
0777.50,017,34187,4.26
0777.50,032,31903,4.26
0777.50,043,30665,4.26
0777.50,052,29987,4.26
0777.50,059,29705,4.26
0777.50,071,30035,4.26
0777.50,085,30951,4.26
0777.50,091,31901,4.26
0777.50,098,32670,4.26
0777.50,105,33406,4.26
0777.50,113,35474,4.26
0777.50,114,35313,4.26
0777.50,142,33655,4.26
0777.50,145,33431,4.26
0777.50,149,32914,4.26
0777.50,172,30268,4.26
0777.50,180,29833,4.26
0777.50,182,29682,4.26
0777.50,184,29655,4.26
0777.50,185,29640,4.26
0777.50,202,30312,4.26
0777.50,215,31699,4.26
0777.50,225,33078,4.26
0777.50,236,35031,4.26
0777.50,239,35561,4.26
0777.50,240,35643,4.26
0777.50,268,33189,4.26
0777.50,270,33476,4.26
0777.50,271,32981,4.26
0777.50,274,32680,4.26
0777.50,278,32215,4.26
0777.50,281,31710,4.26

The top 2 entries in the above list are the ones that were started with 4.26 and finished with 4.32.

What do you think? Does it all look OK?

I'm sure Mike will be interested in this string of results anyway :-).

Cheers,
Gary.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,144
Credit: 129,223,220
RAC: 23,522

RE: What do you think?

Message 77797 in response to message 77796

Quote:
What do you think? Does it all look OK? I'm sure Mike will be interested in this string of results anyway :-).


Yep & it's a nice curve fit! I think I might relax about having to jazz up the algorithm [ to a least squares fit using a Taylor polynomial approximation of sine ] - whew! :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

KSMarksPsych
KSMarksPsych
Moderator
Joined: 15 Oct 05
Posts: 2,702
Credit: 4,090,227
RAC: 0

RE: As promised, here are

Message 77798 in response to message 77793

Quote:

As promised, here are the two files required to get automatic data collection happening on Linux for any hosts you may have an interest in.

Thanks to both Gary and Bikeman. I've been slowly learning shell scripting (I'm in my infancy) and the more examples I can see, the better.

Kathryn :o)

Einstein@Home Moderator

Svenie25
Svenie25
Joined: 21 Mar 05
Posts: 139
Credit: 2,436,862
RAC: 0

Tried somebody to fit the

Tried somebody to fit the curve of the runtimes with Origin? Wich function can I use for fiting? I used the "Sine" function and it looks quite good.

archae86
archae86
Joined: 6 Dec 05
Posts: 2,930
Credit: 3,715,017,990
RAC: 4,727,009

RE: Tried somebody to fit

Message 77800 in response to message 77799

Quote:
Tried somebody to fit the curve of the runtimes with Origin? Wich function can I use for fiting? I used the "Sine" function and it looks quite good.


Check the discussion in this thread

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 1,960
Credit: 334,319,949
RAC: 825,805

RE: RE: I'll have to do

Message 77801 in response to message 77796

Quote:
Quote:
I'll have to do the data collection manually, not having a Linux box to hand.

If you like, I'll collect it for you and email it to you.


Thanks. At about 13 or 14 tasks per day, I should be able to keep up with it manually, but it would be good if you could monitor it as a backup.

The first ten results are through:

909.15,493,26242.63,4.32
909.15,494,26463.16,4.32
909.15,495,26495.86,4.32
909.15,496,26472.66,4.32
909.15,497,26713.83,4.32
909.15,498,26940.11,4.32
909.15,499,26882.64,4.32
909.15,500,27144.52,4.32
909.15,501,27318.17,4.32
909.15,502,26752.42,4.32


Mike's RR6A renders that as:

Number of point pairs used = 10
Minimum runtime in data = 26242.63
Maximum runtime in data = 27318.17
Estimated peak runtime = 27968
Estimated average runtime = 24727
Estimated trough runtime = 22877
Estimated runtime variance = 0.182
Estimated error = 2.6 %

That's looking much lumpier than my previous graph - we'll have to see how it develops over time. Even at under 7 hr 30 min per task, I'm not quite keeping up with the competition, so the downloads have skipped a couple of sequence numbers - but I've got to 475 in cache at the same frequency.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,354
Credit: 48,608,600,023
RAC: 57,717,688

RE: RE: If you like, I'll

Message 77802 in response to message 77801

Quote:
Quote:
If you like, I'll collect it for you and email it to you.

Thanks. At about 13 or 14 tasks per day, I should be able to keep up with it manually, but it would be good if you could monitor it as a backup.

I'll try to post a daily update in case you get sick of doing it manually and also for the benefit of anyone else interested in your progress.

Quote:

The first ten results are through:

Mike's RR6A renders that as:

Number of point pairs used = 10
Minimum runtime in data = 26242.63
Maximum runtime in data = 27318.17
Estimated peak runtime = 27968
Estimated average runtime = 24727
Estimated trough runtime = 22877
Estimated runtime variance = 0.182
Estimated error = 2.6 %

That's looking much lumpier than my previous graph - we'll have to see how it develops over time...

I'm sure you realise that your range of seq#s is very small compared to the period of the cycle so you are really taxing the ability of RR and because it only chose to use 10 point pairs, it obviously threw away as unusable, many others. Once you get to seq#s that are 100 or more below what you started at you will see much better output from RR.

Cheers,
Gary.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,144
Credit: 129,223,220
RAC: 23,522

RE: I'm sure you realise

Message 77803 in response to message 77802

Quote:
I'm sure you realise that your range of seq#s is very small compared to the period of the cycle so you are really taxing the ability of RR and because it only chose to use 10 point pairs, it obviously threw away as unusable, many others. Once you get to seq#s that are 100 or more below what you started at you will see much better output from RR.


This is one of the first I've tried for RR over 900 Hz. I think I'll bite the bullet and do the least squares route, have a play and see whether it is superior to the simpler algorithm presently used. It ought be much less sensitive to problems of close sequence numbers ( phase ). So far, with people's data ( thank you all! ) certainly task groups in narrow bands of phase/sequence seem to be routinely allocated for a given host - so it will be a common question to solve. Also with the least squares methodology one can choose/apply 'weightings' in a fashion that can limit 'the close phase effect' on sine curve matching .... make the weighting inversely proportional to phase difference say and hence suppress that difficulty without having to cull data completely.

Cheers, Mike

I have made this letter longer than usual because I lack the time to make it shorter. Blaise Pascal

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.