Credits granted to large unit are not fair
22 Apr 2007 2:57:29 UTC
Topic 192611
(moderation:
I recently started to receive large WU on einstein@home.
Small WU were taking approx. 20,000 seconds of CPU time and I was granted approx. 55 points for each unit.
Large WU are taking 5 or 6 time longer (100,000 seconds to 120,000 seconds) but I am only granted 3 or 3.5 times the credits.
Can the project team make the credit granting process a bit fairer ? Alternatively, can the end-user control the size of the work unit they wish to receive ?
Thanks !
Credits granted to large unit are not fair
)
Einstein was giving about 30-40% higher credit than most of the other major projects, which is why the credit was sharply cut for s5r2. It's already been boosted once, so another kick might be possible in the future if it's still too low, but you're not going to see the same level of credit as with s5r1. One of hte other s5r2 threads has a link to a credit comparison graph that shows einstien out crediting almost every other project.
RE: Large WU are taking 5
)
Your computers are hidden, but if you are running Windows on AMD, then read this.
cu,
Michael
RE: Einstein was giving
)
There's a bit of a flaw in the logic being used for cutting the credit though. The S5R1 standard-issue app was a well-optimized app. This means it is going to perform faster than an unoptimized app. All Einstein users used this optimized app. Over on SETI, where it appears that there is the most "disparity", only some optimization is put into the standard app. Those of us who use Simon Zadra's ("Chicken Coop") app have to make the choice to do so, downloading it and putting it in the right place.
S5R1 was already giving me less credit per CPU minute than the optimized SETI app. That is not conjecture, but undisputable fact. I wanted to break the 100,000 credit threshold here at Einstein before the end of S5R1. I started dedicating my AMD host to Einstein around the end of February. If you look at my host stat graph, what happened then?
here's my AMD host's BOINC-wide stat page at BOINCStats
You'll note that I immediately started losing ground in the world position. If I was getting "too many" credits, I don't think that would've happened, but perhaps I'm wrong...
A 49.10 credit unit at SETI took 5400 seconds today. That comes out to be 0.545555555 cr/min. A 53.50 credit unit here at Einstein took rougly 7900 seconds, for 0.406329114 cr/min. For every 1 hour of time on each, SETI has an 8.3536 credit advantage per cpu core from where I stand (or sit, as it were) ;-)
I would agree with what Alinator stated over in the other thread, that other projects need to work on bringing themselves up to Einstein's level rather than Einstein dropping down again. You can see those comments here. In my opinion, SETI too needs to bring its' credits in line with Einstein, and according to what I'm reading, that's what they were trying to do with the beta apps. With this drop by Einstein, SETI will be faced with the choice of either staying where they are or lowering theirs as well, which will likely cause an eruption of tempers...
Finally, as I stated some time ago and seem to have been largely ignored (I know, they said that they're busy), in my opinion R2 needs to be in beta test phase to work on getting the optimization back into the new app. Otherwise this is simply a beta test using somewhat unwilling participants, IMO.
Brian
Personally, I'm not
)
Personally, I'm not unwilling... and from what I read on the other threads, there were quite a few people enthusiastic about S5R2 starting. If you ask me, it's going to work out soon enough- the science app will get more efficient again, and if we really get too little credit, I'm sure that will be taken care of, too. So give the project staff a break, I think they've been doing great work so far!
Just my 2 cents.
RE: Personally, I'm not
)
I understand where you're coming from. Really I do. However you have been fortunate in that you are not getting client errors. I didn't mention that in my prior post in this thread, but I am seeing systems here and there showing errors. I found one host that was doing fine with S5R1, but the instant that computer started attempting S5R2, the workunits crashed. That's been going on for a few days now, so that particular person may not have noticed that every result that they were downloading was now erroring out in no more than 4 minutes.
I am a software developer. I do not have the same kind of skills as the people working on this project do, but I know enough to know that the crashes that I'm seeing indicate that this software needed/needs to be tested more thoroughly. I see that you have now seen the post by Akos. This is the beta test page: Einstein Beta. In my opinion, this is where S5R2 needs to be right now based on what I'm seeing and in conjunction with Akos' comment.
Brian
Credit balancing is only with
)
Credit balancing is only with respect to the current official app. The existance of a faster unofficial application is not and should not be a factor, except in that when the official one rolls in the changes from the faster one that the credits per WU needs to be decreased.
What you seem to be advocating is a fixed credit per WU and then boosting credits per hour be releasing more and more optimized applications. This would lead to abuse as projects would simply release an extremely bad application with thier new WU types, and then quickly release a much better one to atract credit whores.This would completely undermine the goal of credits being used to compare cross project performance.
RE: Credit balancing is
)
"Seem" is a tad far from reality... I'll attempt explaining it again in the hopes that what I'm saying will be clearer to you.
moderate to heavy optimization in its code.
*The standard / "official" SETI app has light optimization in the code.
You cannot compare a more heavily optimized application to a significantly lesser one and then adjust down. It just doesn't make any sense. Let's say that SETI optimizes their app further, incorporating proven routines from the "unofficial" applications. Do they then knock their credit down even more? In turn does Einstein, now realizing that they are giving "too much" knock theirs down again? How many iterations of that will it take to get to "parity"? What happens if several projects release new applications that are significantly slower and they decide that they are not going to attempt to speed it up? Does Einstein then reduce downward because of them too?
Alinator made the suggestion that Einstein work with the rest of the BOINC community to raise the credits of the other projects to be in line with Einstein rather than to continue to devalue the Einstein work. This proposal is just a positive spin to the same objective, i.e. elevating other credits rather than devaluing Einstein credits. Fundamentally, what Alinator said, and what I also agree with, is looking at it "half full" instead of "half empty", to borrow a well-overused cliche. Same goal, but with a different, and in my opinion, more positive approach.
The first priority though should be to fix the bugs / crashes. This is what I view as what should be the new top priority for the application. In the small amount of time that I spent dredging through some results, I'm noticing crashes on both newer and older AMD platforms, and so far only on older Intel platforms. I haven't cross-referenced the OS involved, but I don't think I've seen any crashes on Linux.
To go back to what you said above:
I would suggest that Einstein has already released an extremely bad application with S5R2. I'm sure they did their best in the time they had, but this application was CLEARLY not ready to be in production. I am currently showing a 10-fold increase in completion times, for approx 3-4 times the credit, assuming that the application does not crash.
P.S. - you'll note that I didn't care so much about the value of S5R1 vs. SETI, but the more I look at this, I do care about S5R2 vs. SETI. This is coming from someone who is seriously considering stopping SETI anyway because I'm not seeing much scientific value these days in that project...
@Brian: LOL.... It's
)
@Brian:
LOL....
It's gratifying that someone finally picked up on the concept that the intrinsic value of the science performed must be assumed to be equal, regardless of what project you're talking about, if any credit system is to be applicable across all of them. IOW, scientifically speaking no one project is more important than any other. ;-)
Once you accept that, then you can pick one or more projects to use as standard to measure the others by, and thus be able to set the rate factors so that on a given machine the rate is the same for any project you run on it. This has always been the goal of BOINC from the start.
Arguements about project side credit whoring (a less genteel way of saying inflation) while possible, is unlikely due to the fact that its would seem to be contrary to the goals of a science project and thus put it fundamental merits and objectivity into question in the eyes of the participant community (contrary to a rebuttal to these same questions on another projects Q&A forum).
On the other hand, the deflation of the value of the work which is being proposed makes me the user think; Why is the work I'm doing now worth less, than before? It must be it's less important. Therefore why should I bother with it, especially if it takes longer to run.
In economic terms, runaway devaluation leads to crashes and depressions. Extrapolated into the BOINC world it leads to people thinking, this a waste of my CPU time and my cold hard cash to run it, and then drop out.
The only thing I have proposed is that EAH and SAH would be my choices for the 'gold' standard for the reasons I mentioned before. It does not mean that those standards shouldn't be given a reality check review periodically, or that they are forever carved in stone.
Alinator
It just occurred to me that ultimately infinite devaluation ends up being just counting the number of WU's you ran and listing the number of CPU hours you burned. Certainly a valid way of measuring participation, but hardly conducive to running a mix of projects. :-)
RE: Credit balancing is
)
No, what I'm saying is that by the end of S5R1 my P4 for example had an average credit rate of about 14 per hour, which was essentially the same for S5RI, and unless your telling me that the work we will do on S5R2 is somehow not as valuable as before should pay about 14 credits per hour on that machine.
Also, the EAH team made the effort to look at cross project parity during S5R1 at the same time they were improving the performance of the app and made adjustments accordingly to avoid the inflation you mention. BTW this required reducing the basis of the science's value significantly compared to S4.
Alinator
RE: Personally, I'm not
)
This is certainly our intention. This week I will do some work on the part of the scheduler that sends shorter WU to slower hosts. That should help address part of the problems that some people have experienced, with WU that are too long for their hosts.
Cheers,
Bruce
Director, Einstein@Home