CPU Time versus Credit

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3,516
Credit: 461,673,562
RAC: 41,504

RE: Looking throughout this

Message 97441 in response to message 97440

Quote:

Looking throughout this thread, I suddenly understood that newer WU's give us almost equal credits but take each time more time to complete a WU for the current search than for the previous seaches. This means that new search WUs contain less data (in cobblestones) or (and I slowly come to believe in that) newer clients are less and less optimized than previous ones, thus making less useful work with each new version or run.

For example, let's assume that modern Core Quad ten times more powerful than first generation P4. Given that, we should receive ten times more credits using the same app. And that is true, because P4 takes 10 times longer to proccess the same WU. It is ok, but....
If we compare S4 WUs with new S5GCE WUs, we will found that the same processor gives us more credits per hour with older app than with the new one.
What does it mean? And why does this happen? May be 2 years ago we used more scientifically-powerful applications? Or may be new programmers can't reach the same level of optimization in new apps? It will be interesting to listen for any opinion and any interesting explanation.

Hi!

Even tho I'm just a volunteer developer, let me try to shed some light on this:

1) Runtime per workunit:

You cannot directly compare WU's of different runs, like S4 and S5GCE. It's true, if you would run 2005 vintage WUs on today's hardware, it would probably finish within minutes. What does that mean for WU design? You have to put more science (e.g. points of the sky to look at) into a single WU because you have to have a sane ratio between runtime and the amount of data that has to be downloaded. Otherwise users will complain about high network traffic and download servers might crumble under the load.

Take ABP2, for example: The original units just executed "too fast", so 4 workunits were later combined into a single new type of WU, taking 4 times the time to complete, of course. So is the new app less efficient because it takes longer? No... it does the same amount of science per unit time

2) Optimization

The current run S5GCE uses a whole new search algorithm ("Global correlation") but currently still maintains the code that was contributed directly or inspired by Akos and others. This part of algorithm is highly optimized.
It is natural that the primary focus in this "Engineering" run is on correctness of the search method. Note that there are two ways to waste computing power: by poorly optimized code or by code that simply does not work as intended. But optimization of the code IS on the agenda, just like in previous runs.

Happy crunching
HB

Whiskymania
Whiskymania
Joined: 2 Sep 05
Posts: 8
Credit: 10,030,266
RAC: 0

Hi again May I ask a

Hi again

May I ask a Question?
Look at the table:
170923963 72615133 15 Apr 2010 21:00:44 UTC 29 Apr 2010 15:37:09 UTC Over Success Done 10,051.90 53.32 160.00
170921115 72613930 15 Apr 2010 13:19:21 UTC 29 Apr 2010 6:31:49 UTC Over Success Done 33,038.58 74.83 250.69

These are two WUs from Bikeman's crunching machine: http://einsteinathome.org/host/1767091

Another example, taken from DanNeely
172253024 73181659 26 Apr 2010 2:14:05 UTC 29 Apr 2010 18:13:27 UTC Over Success Done 13,718.34 68.06 160.00
172236900 73174694 25 Apr 2010 13:03:34 UTC 29 Apr 2010 18:13:27 UTC Over Success Done 28,050.90 74.83 250.69
The machine is: http://einsteinathome.org/host/1694313

So, thats all that I'm talking about. On Bikemans machine S5GCE takes more than 3 times longer and on DanNeely's machine S5GCE takes arround 2 times longer than ABP WUs.
It's interresting to see, that under Windows it seems to be the same fact as on OS X like my own machine: S5GCE is taking longer than ABP.

I agree on the fact, that runtimes are depending on code, package and all that stuff - but aren't these (above) differences obvious, are they?

Have a nice day
Markus/CH

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3,516
Credit: 461,673,562
RAC: 41,504

RE: Hi again May I ask a

Message 97443 in response to message 97442

Quote:

Hi again

May I ask a Question?
Look at the table:
170923963 72615133 15 Apr 2010 21:00:44 UTC 29 Apr 2010 15:37:09 UTC Over Success Done 10,051.90 53.32 160.00
170921115 72613930 15 Apr 2010 13:19:21 UTC 29 Apr 2010 6:31:49 UTC Over Success Done 33,038.58 74.83 250.69

These are two WUs from Bikeman's crunching machine: http://einsteinathome.org/host/1767091

Another example, taken from DanNeely
172253024 73181659 26 Apr 2010 2:14:05 UTC 29 Apr 2010 18:13:27 UTC Over Success Done 13,718.34 68.06 160.00
172236900 73174694 25 Apr 2010 13:03:34 UTC 29 Apr 2010 18:13:27 UTC Over Success Done 28,050.90 74.83 250.69
The machine is: http://einsteinathome.org/host/1694313

So, thats all that I'm talking about. On Bikemans machine S5GCE takes more than 3 times longer and on DanNeely's machine S5GCE takes arround 2 times longer than ABP WUs.
It's interresting to see, that under Windows it seems to be the same fact as on OS X like my own machine: S5GCE is taking longer than ABP.

I agree on the fact, that runtimes are depending on code, package and all that stuff - but aren't these (above) differences obvious, are they?

Have a nice day
Markus/CH

Hi Markus,

Ok, but comparing the runtime of ABP2 and S5GCE WUs is a bit like comparing apples to oranges: It's not that they are different versions of the same thing, they are doing completely different searches: ABP2 is analysing data from the Arecibo radio-telescope (several minutes in a row) to find pulsars. S5GCE WUs are analysing months worth of data of the LIGO gravitational wave observatories.

But the hosts that you picked are very interesting indeed. This examples shows how impossible it is to award credits in a way that is "fair" across operating systems (and compiler versions), CPU models and apps.

Dan's running a Windows host, mine is under Linux. Dan's i7 is using hyperthreading: 4 physical cores run 8 WUs in parallel. Mine is a Core2 Quad core and executes only four WUs in parallel. Every scheme that would be "fair" for Dan's host (in the sense that credit/runtime is the same for both searches) would not be fair for my host, and vice versa.

Happy crunching
HB

Whiskymania
Whiskymania
Joined: 2 Sep 05
Posts: 8
Credit: 10,030,266
RAC: 0

RE: Hi Markus, Ok, but

Message 97444 in response to message 97443

Quote:

Hi Markus,

Ok, but comparing the runtime of ABP2 and S5GCE WUs is a bit like comparing apples to oranges: It's not that they are different versions of the same thing, they are doing completely different searches: ...

Dear Bikeman

Ohh, appologies - I didn't new that, I just had the eye on how many hours must be crunched until a WU ist finished and how many credits this pays ;-)

I think both are good for what they are ment for and it doesn't count if they are oranges or apple or even bananas or lemons - does it?
If it does - I would like to invite you to explain in detail, WHAT they are exactly crunching (the apple and oranges and ...) and how the "bill" is made from that, will you?
Besides that I have to point out that I'm not a scientist and will probabely need some more basic stuff (knowledge).

I hope you understand me the right way (the "sound" of this post) - thats what I would want and is intended ;-)
... and that english is not my mothertongue ;-)

Chers and have a nice WE!
Markus/CH

Whiskymania
Whiskymania
Joined: 2 Sep 05
Posts: 8
Credit: 10,030,266
RAC: 0

May I ask another

Message 97445 in response to message 97443

May I ask another question?

Why is it not possible to choose applications in "Run only the selected applications" ?
It's simply not possible to really "select" by myself ;-)

Cheers
Markus/CH

Jord
Joined: 26 Jan 05
Posts: 2,952
Credit: 5,725,162
RAC: 316

S5Rx is always available, ABP

Message 97446 in response to message 97445

S5Rx is always available, ABP isn't.

Whiskymania
Whiskymania
Joined: 2 Sep 05
Posts: 8
Credit: 10,030,266
RAC: 0

RE: S5Rx is always

Message 97447 in response to message 97446

Quote:
S5Rx is always available, ABP isn't.

Thx for this information. But shouldn't it be my own descision, which of them all I would like to run, even when that implies, that there is maybe not enough work for my descision?

Shouldn't it be, in a more far perspective, like the descision which project at all should be supported?

Have a nice day
Markus/CH

transient
transient
Joined: 3 Jun 05
Posts: 62
Credit: 114,218,425
RAC: 25,941

You can opt out for APB. That

You can opt out for APB. That may not seem like it, but try editing the preferences. I see a checkmark that I can remove. I assume it is working. I haven't actually tested it.

Whiskymania
Whiskymania
Joined: 2 Sep 05
Posts: 8
Credit: 10,030,266
RAC: 0

RE: You can opt out for

Message 97449 in response to message 97448

Quote:
You can opt out for APB. That may not seem like it, but try editing the preferences. I see a checkmark that I can remove. I assume it is working. I haven't actually tested it.

Well - I would like to opt out S5GCE ;-)
Certainly it works, why else should this option be there (?)

Thx for your idea anyway ;-)

Have a nice day
Markus/CH

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3,516
Credit: 461,673,562
RAC: 41,504

RE: RE: You can opt out

Message 97450 in response to message 97449

Quote:
Quote:
You can opt out for APB. That may not seem like it, but try editing the preferences. I see a checkmark that I can remove. I assume it is working. I haven't actually tested it.

Well - I would like to opt out S5GCE ;-)
Certainly it works, why else should this option be there (?)

Thx for your idea anyway ;-)

Have a nice day
Markus/CH

This has to do with the main scientific focus of the project (LIGO data processing), and there are also constraints I guess because of the grants that E@H received, e.g. from NSF. If you get grants for growing oranges, you can grow apples with spare ressources, but you shouldn't be able to switch completly to growing apples...at the end you have to show some nice oranges to the funding bodies :-)

CU
HB

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.