Using 7G Ready Reckoner as downloaded about 6:00 UTC October 14, running with the twiddle factors at default (D=.00029, G=.42, Q=10)
On a Q6600 host running 2.8 GHz with slowish RAM settings:
For 6.04 release on 96 tasks at 541.75 RR estimates:
Peak = 35466 | Average = 31635 | Trough = 29448 | Variance = 0.17
For 6.05 release on 56 tasks at 541.95 RR likes
Peak = 30760 | Average = 26369 | Trough = 23863 | Variance = 0.224
(I suppressed the four 6.05 outlier times previously mentioned)
Each of these provides pretty decent sampling over a full cycle (from near the second trough to near the third trough). The average figure should estimate overall output improvement--this says 20%. The new trough is predicted to be 81% of the old trough, while the new peak is predicted to be almost 87% of the old peak. So this model suggests much bigger proportional improvement in the trough than peak (as previously suggested). Hence the temptation to estimate overall improvement from the nice broad trough alone is a false path.
Similarly, the noise level on my host (even without the four result transient) is considerable, so many samples are needed for a decent estimate.
The posted images use RR's Single Cycle option to map all points onto a single cycle.
(images downsized to 500 pixels wide, and somewhat contrast-increased in Photoshop)
I'm less confident about the modelling, or that box or both! :-)
The single cycle display is great when the cycle length is working right, but the moment you suspect it of being wrong, I think it wise to switch to multi-cycle display. If the available data cover a long enough range of sequence numbers, the multi-cycle display should make it obvious if the problem is with the estimated cycle length. With too short a range of sequence numbers, there is not much that can be done.
I've done some digging and can confirm that if your WU uses a skygrid file of frequency f_skyHZ (which will always be a multiple of 10Hz), the runtime period will always be
P ~ ceil(0.000291 x f_sky^2)
Which is exactly the value Mike got empirically.
Now, the skygrid-frequency itself will be close to
ceil((f+delta)/10)*10
where f is the frequency taken from the WU name and delta is really complex to calculate exactly but mostly on the order of ~ 0.1...0.2Hz.
So if the workunit frequency is not close to a multiple of 10Hz you are safe, round that frequency upwards to the next multiple of ten, square, multiply with 0.000291 , round upwards and that's the period you are looking for.
Or better use the Ready Reckoner with the indicated constant. So nothing new really, just a confirmation from an analysis of the WU generation process that the new constants for S5R4 were just right. Well done!
Pretty close to my estimate. cycle length(freq)=0.42+.00029*(ceiling(freq,10)+1)^2
Actually I did not specify a ceiling function on the final result, but had I done so my curve-fit function would have matched your finding exactly for the actual frequencies at hand on my little fleet save the last one:
Where the pas column is mine as published, pasc that with a final ceiling imposed, and bikeman is yours (I set your delta to 0.15, and put in the squaring you mentioned in text).
The discrepant 292 differs by the barest of margins--my calc before the ceiling function actually ends in .00029, so taking the ceiling rounds it up to a much larger discrepancy than without.
If one peeks under the covers before the last ceiling function covers up small differences, one finds that down at 182 the estimates differ by almost 0.5 with mine higher. So it is likely that for really low frequencies not present in my data they will differ. At the 997 level the raw estimates are essentially identical: to five significant digits 291.00000 vs. 291.00029.
However that is because 997 is a crossover point. At higher frequencies my raw estimate is slightly lower than yours, though even as high as 1305 they are so close that with the ceiling function they are often the same.
The biggest discrepancy, by far, comes for high frequencies very near the 10 Hz step point. I failed to put in a fiddle factor (though I was aware of the problem), so mine steps right at the 10 Hz boundary while yours steps a little before. Those steps get big up above 1000 Hz, so the resulting discrepancy at, for example, 1509.95 is that you predict a cycle length of 673, while I only 663, though for 1511.00 we differ by much less.
In order actually to use it in my copy of Excel, I just had to honor Excel's ceiling function syntax (they want you to say 1 explicitly, as you did for 10).
The results post the final ceiling function are the same as your earlier one for my small collection of actual observed cycles. Peeking behind that ceiling, they are closer to each other than to my original guess.
As to the fiddle 10 Hz boundary question, sadly my modest collection of data includes no cases closer to a boundary than some data I have at 530.25. I can look at the two "possibilities" for that case, and affirm it is past the step-up, but that was really not in doubt.
For a moment I thought I had something when I saw data logged from these frequencies:
Sadly it was in late June 2008, so on the previous project.
Thanks for the investigation. Looking at things a bit backwards, the fact my model fit to an improvised structure fits that closely raises my guess that you have the right of it--I'd advise people to use yours in preference to mine without reservation.
Pending direct observation of confirmation in the fiddle region, any code might best warn users that the estimate in the very small region in any doubt may be unreliable. The region is small, but the error is not--especially at high frequencies.
Using 7G Ready Reckoner as
)
Using 7G Ready Reckoner as downloaded about 6:00 UTC October 14, running with the twiddle factors at default (D=.00029, G=.42, Q=10)
On a Q6600 host running 2.8 GHz with slowish RAM settings:
For 6.04 release on 96 tasks at 541.75 RR estimates:
Peak = 35466 | Average = 31635 | Trough = 29448 | Variance = 0.17
For 6.05 release on 56 tasks at 541.95 RR likes
Peak = 30760 | Average = 26369 | Trough = 23863 | Variance = 0.224
(I suppressed the four 6.05 outlier times previously mentioned)
Each of these provides pretty decent sampling over a full cycle (from near the second trough to near the third trough). The average figure should estimate overall output improvement--this says 20%. The new trough is predicted to be 81% of the old trough, while the new peak is predicted to be almost 87% of the old peak. So this model suggests much bigger proportional improvement in the trough than peak (as previously suggested). Hence the temptation to estimate overall improvement from the nice broad trough alone is a false path.
Similarly, the noise level on my host (even without the four result transient) is considerable, so many samples are needed for a decent estimate.
The posted images use RR's Single Cycle option to map all points onto a single cycle.
(images downsized to 500 pixels wide, and somewhat contrast-increased in Photoshop)
OK, Winternight's data looks
)
OK, Winternight's data looks like thus:
and Pete's also looks good. Meaning I think the cycle length is working with the new parameters. But with some of Gary's ( Host # 1608085 ):
I'm less confident about the modelling, or that box or both! :-)
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
More from the 1048 series,
)
More from the 1048 series, V6.05
1048,855,22944.44
1048,854,22998.95
1048,853,23221.16
1048,852,23221.25
Wasn't going to post so soon, but the similarity in times for last two caught my eye.
RE: I'm less confident
)
The single cycle display is great when the cycle length is working right, but the moment you suspect it of being wrong, I think it wise to switch to multi-cycle display. If the available data cover a long enough range of sequence numbers, the multi-cycle display should make it obvious if the problem is with the estimated cycle length. With too short a range of sequence numbers, there is not much that can be done.
I've done some digging and
)
I've done some digging and can confirm that if your WU uses a skygrid file of frequency f_skyHZ (which will always be a multiple of 10Hz), the runtime period will always be
P ~ ceil(0.000291 x f_sky^2)
Which is exactly the value Mike got empirically.
Now, the skygrid-frequency itself will be close to
ceil((f+delta)/10)*10
where f is the frequency taken from the WU name and delta is really complex to calculate exactly but mostly on the order of ~ 0.1...0.2Hz.
So if the workunit frequency is not close to a multiple of 10Hz you are safe, round that frequency upwards to the next multiple of ten, square, multiply with 0.000291 , round upwards and that's the period you are looking for.
Or better use the Ready Reckoner with the indicated constant. So nothing new really, just a confirmation from an analysis of the WU generation process that the new constants for S5R4 were just right. Well done!
CU
Bikeman
RE: the runtime period
)
Pretty close to my estimate.
cycle length(freq)=0.42+.00029*(ceiling(freq,10)+1)^2Actually I did not specify a ceiling function on the final result, but had I done so my curve-fit function would have matched your finding exactly for the actual frequencies at hand on my little fleet save the last one:
_freq_ _pas_ pasc bikeman 182.90 11.00 11.0 11.0 291.65 26.69 27.0 27.0 431.00 56.82 57.0 57.0 530.30 85.30 86.0 85.0 541.00 88.46 89.0 89.0 744.50 164.0 164. 164. 776.00 177.3 178. 178. 997.00 291.0 292. 291Where the pas column is mine as published, pasc that with a final ceiling imposed, and bikeman is yours (I set your delta to 0.15, and put in the squaring you mentioned in text).
The discrepant 292 differs by the barest of margins--my calc before the ceiling function actually ends in .00029, so taking the ceiling rounds it up to a much larger discrepancy than without.
If one peeks under the covers before the last ceiling function covers up small differences, one finds that down at 182 the estimates differ by almost 0.5 with mine higher. So it is likely that for really low frequencies not present in my data they will differ. At the 997 level the raw estimates are essentially identical: to five significant digits 291.00000 vs. 291.00029.
However that is because 997 is a crossover point. At higher frequencies my raw estimate is slightly lower than yours, though even as high as 1305 they are so close that with the ceiling function they are often the same.
The biggest discrepancy, by far, comes for high frequencies very near the 10 Hz step point. I failed to put in a fiddle factor (though I was aware of the problem), so mine steps right at the 10 Hz boundary while yours steps a little before. Those steps get big up above 1000 Hz, so the resulting discrepancy at, for example, 1509.95 is that you predict a cycle length of 673, while I only 663, though for 1511.00 we differ by much less.
Upps, yes, I forgot the
)
Upps, yes, I forgot the square in the formula, thanks for noticing, I corrected this in my post.
I'll try to post a formula for the fiddle factor (rather a fiddle function) later when I find time to dig deeper :-)
CU
Bikeman
OK, here's my approximation:
)
OK, here's my approximation: (for S5R4 only, should be very close)
cycle_length(freq)=ceiling(.00029096*(ceiling( fiddle(freq)+freq , 10))^2)
where
fiddle(freq) = 0.11501 + freq*1.05e-4
so
cycle_length(freq)=ceiling(.00029096*(ceiling( 0.11501 + freq*(1+1.05e-4) , 10))^2)
Best verified by comparing the actual sky-grid-file frequency for workunits with a frequency close to a multiple of 10Hz to the prediction
f_sky(freq)=ceiling(fiddle(freq)+freq , 10)
Does this fit your data?
CU
Bikeman
RE: cycle_length(freq)=ceil
)
In order actually to use it in my copy of Excel, I just had to honor Excel's ceiling function syntax (they want you to say 1 explicitly, as you did for 10).
The results post the final ceiling function are the same as your earlier one for my small collection of actual observed cycles. Peeking behind that ceiling, they are closer to each other than to my original guess.
As to the fiddle 10 Hz boundary question, sadly my modest collection of data includes no cases closer to a boundary than some data I have at 530.25. I can look at the two "possibilities" for that case, and affirm it is past the step-up, but that was really not in doubt.
For a moment I thought I had something when I saw data logged from these frequencies:
1109.70
1109.75
1109.80
1109.85
1109.90
1109.95
1110.00
1110.05
1110.00
1110.05
1110.10
1110.15
Sadly it was in late June 2008, so on the previous project.
Thanks for the investigation. Looking at things a bit backwards, the fact my model fit to an improvised structure fits that closely raises my guess that you have the right of it--I'd advise people to use yours in preference to mine without reservation.
Pending direct observation of confirmation in the fiddle region, any code might best warn users that the estimate in the very small region in any doubt may be unreliable. The region is small, but the error is not--especially at high frequencies.