In the BOINC Survey, when asked for reasons to join a project,
the answer "because I get more credits there than in other projects" (translated from German) received the lowest numbers of votes among all alternatives. The answer "Because the Research is important and beneficial to the public" got more than 20 times as many votes.
To be fair, the "cross project credit-aware" users will probably have a more than proportional share in the overall project performance, but still I wonder whether this cross-project credit parity thing is too big an effort for a very, very small (but outspoken) minority of the user base, confusing/annoying quite a lot of people at the same time.
CU
Bikeman
I've repeatedly said that it does not appear to be very large of an issue.
Not only that, but there is no guarantee against a "more credits" vote actually being cast by someone who really doesn't feel that way and their agenda was simply to make the figure look bigger.
I'm NOT saying that did happen, but I am saying that there does not appear to have been any mechanism in place to prevent it from happening. The survey was all "honor system", and thus should not be taken as 100% gospel.*
* Yes, I know that also means that people who were/are in it for more credits could've also chose another option to hide their real intent as well...
Also, one of the projects that has recently been highly demonized for giving "too much" was Cosmology. Right now, Cosmology's work rate (and thus credit rate as well) has essentially imploded. Take a look at my productivity there. They took the route of "credits were generous before, so now during this time of problems, nobody gets any credit, so it all evens out". I'm still processing some tasks there, but turnaround time is real slow because Homogeneous Redundancy is enabled there and it appears as though the upheaval completely uprooted a large group of X2 3800+ systems that I was routinely paired up with, so I have tasks that are languishing waiting on another system in my same HR class to come along. On top of that, the project decided to do server-side aborts last week, but not in the "friendly" way (in-progress will still get counted, only non-started get aborted), trapping some tasks in this limbo-land of having 1 success, 1 no reply, 1 "didn't need" or "redundant", and 1 "unsent"...with a quorum of 2 (?!?!?!?!)
Trying to think this through from all perspectives, this issue did not seem to have been an issue before Mr. Anderson brought it up--is that correct?
No, the issue has been around for quite some time, but it is routinely made out to be more severe of an issue than it really is. The general "fear" goes like this:
*Project X needs additional participants to keep it running.
*Project X realizes that a certain segment of users, the competitive users, might be more inclined to participate in their project if they are offered more credit than what Project Y offers.
*Project X increases credit.
*The competitive users flock to Project X, leaving Project Y to figure out whether or not they have enough remaining participants to continue operating within their desired parameters.
*Project Y may then decide that they do not have enough participants, so, learing from Project X's example, Project Y increases their credit rate to more than Project X's credit rate.
*Credit war ensues.
*Projects A-W and Z notice the war and decide that they need more participants too, so they join in and thus you have ever-escalating credit rates.
This scenario, while possible, has just never materialized.
I do not know why this is of such great importance to Mr. Anderson, given that it hasn't happened, and doesn't appear likely to happen given that the current project admins over the past few years have not even responded to the fact that the chart at BOINC Combined Statistics showed them lower than SETI and Einstein and they still did not take action to change that...
The whole thing is akin to "tilting at windmills", in my opinion...
*The "resend_lost_results" (or is that "missing" results?) needs to be fixed to where it is not such a drag on server performance. In lieu of that, better (transaction-based) communication needs to be built into the BOINC client and server software so that "ghost results" are no longer generated.
*Give users the ability to abort tasks via the project web pages. This goes hand-in-hand with the ghost result problem as well as gives a user remote-cancellation capabilities in the event of being away from the host(s) in question.
If you remove the idea of cross project credit parity, therefore surely the next step is to remove credits and all other measuring systems totally. And when I say totally I mean just that no accounting for work done even within the project. No counting of units done, because 10 S5R4's (oranges) do not equal 10 S5R3's (apples). No totaling of time done etc. etc.
'... therefore surely the next step ...' is an incorrect linkage without logical basis. We are not discussing credit abolition, and I don't know anyone who has proposed that. Credit could be abolished tomorrow, with the attendant problems you mention, regardless of how the cross-project issue is resolved.
Quote:
But remember there are a lot of competitive people out there ....
People can apply their own cross-project metrics to suit whatever criteria can be imagined, without bounds! Thus they can avoid the trials of continually having to seek validation in various forums, and those who don't have that focus need not respond. It is precisely this type of angst generated by the cross project credit issue that ought to be eliminated. Perhaps the time has come for project credit policies to be dominated by their respective loyal contributors, and not by those who may or not be there next week depending upon the price of some perpetual negotiation. Personally I think it is time the bluff was called on this negative type of competitive behaviour. I say negative because it retards, not advances, the underlying science efforts through diversion of energies. E@H ( like many projects ) is definitely a very co-operative venture of quite significant scientific importance, and ought not be hampered by those whose competitive energies would be better discharged elsewhere if they are incapable of subjugating themselves to common causes.
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
*The "resend_lost_results" (or is that "missing" results?) needs to be fixed to where it is not such a drag on server performance. In lieu of that, better (transaction-based) communication needs to be built into the BOINC client and server software so that "ghost results" are no longer generated.
*Give users the ability to abort tasks via the project web pages. This goes hand-in-hand with the ghost result problem as well as gives a user remote-cancellation capabilities in the event of being away from the host(s) in question.
I did not know that resend_lost_results was broken (this code was written by David Anderson and me). It is expensive to make the necessary database query, and this is simply unavoidable, so some projects can't afford that database strain.
The idea of a transaction-based communication is interesting. At least in principle one could wrap the entire scheduler interaction into a transaction and then try and get a positive acknowledgment from the BOINC client before committing it. I'm not sure if the current design/code would permit this.
I would be interested in understanding better how ghost results are generated.
By the way, looking at the current E@H scheduler log I see:
which means that less than six percent of results are re-sent. And I imagine that this ratio will become smaller as the effects of our server upgrade two weeks ago fades away.
I did not know that resend_lost_results was broken (this code was written by David Anderson and me). It is expensive to make the necessary database query, and this is simply unavoidable, so some projects can't afford that database strain.
I wouldn't say that resend_lost_results was broken. It works well at SETI Beta, and it has worked well here except for the brief interlude at the beginning of August when it was re-sending results than couldn't be processed (now fixed).
There is an ongoing problem at SETI where resend_lost_results has been deliberately switched off by the project administrators because they found that the database query load was unsustainable in their particular environment.
Quote:
The idea of a transaction-based communication is interesting. At least in principle one could wrap the entire scheduler interaction into a transaction and then try and get a positive acknowledgment from the BOINC client before committing it. I'm not sure if the current design/code would permit this.
I would be interested in understanding better how ghost results are generated.
In my experience, there are two ways that 'ghost results' come to be generated:
1) Sporadically, when a communications problem means that a client-scheduler interaction doesn't come to a clean conclusion. Specifically, the server scheduler allocates a task to the host and updates the WU and task records accordingly: but the message telling the client about the task is not received, typically because of an http error or timeout - most common at times of high network traffic (SETI: outage recovery. Einstein: old run cleanup/new run initialisation). Transaction processing would help here: or maybe if the WU/tasks tables were not updated to reflect the task allocation until after the client acknowledges receipt of the task allocation message. Of course, that could cause the opposite problem (what's the opposite of a ghost?): the client receives the task instructions and starts processing it, but the server fails to receive the acknowledgement and thinks the allocation has failed. Cue a surprise when the results of the completed task are reported.
2) Systemically, when a change in server code creates a bug (or reveals a pre-existing bug) which means that every scheduler contact by a specific class of users fails and results in a ghost.
I have seen two major outbreaks of this class of mass haunting: in May 2007 (SETI only - trac [trac]#194[/trac]), and August 2008 (SETI and Einstein - trac [trac]#713[/trac]). Both instances were tracked down to problems with the Anonymous Platform mechanism.
I did not know that resend_lost_results was broken (this code was written by David Anderson and me). It is expensive to make the necessary database query, and this is simply unavoidable, so some projects can't afford that database strain.
I wouldn't say that resend_lost_results was broken. It works well at SETI Beta, and it has worked well here except for the brief interlude at the beginning of August when it was re-sending results than couldn't be processed (now fixed).
There is an ongoing problem at SETI where resend_lost_results has been deliberately switched off by the project administrators because they found that the database query load was unsustainable in their particular environment.
Cosmology also switched it off due to the high number of tasks in people's caches. Since it has to build such a large query list, I think any project that ends up with a fairly large result table to have to look through would end up having similar problems... Due to the 3-5X increase in processing time at Cosmology, I would imagine that they could turn the functionality back on now... They have had several instances where people are stating that they see results from the web page that are not on their local machine(s), thus they have had "Class 5 full roaming vapors" (aka "ghosts")
Quote:
Quote:
The idea of a transaction-based communication is interesting. At least in principle one could wrap the entire scheduler interaction into a transaction and then try and get a positive acknowledgment from the BOINC client before committing it. I'm not sure if the current design/code would permit this.
I would be interested in understanding better how ghost results are generated.
In my experience, there are two ways that 'ghost results' come to be generated:
1) Sporadically, when a communications problem means that a client-scheduler interaction doesn't come to a clean conclusion. Specifically, the server scheduler allocates a task to the host and updates the WU and task records accordingly: but the message telling the client about the task is not received, typically because of an http error or timeout - most common at times of high network traffic (SETI: outage recovery. Einstein: old run cleanup/new run initialisation). Transaction processing would help here: or maybe if the WU/tasks tables were not updated to reflect the task allocation until after the client acknowledges receipt of the task allocation message. Of course, that could cause the opposite problem (what's the opposite of a ghost?): the client receives the task instructions and starts processing it, but the server fails to receive the acknowledgement and thinks the allocation has failed. Cue a surprise when the results of the completed task are reported.
2) Systemically, when a change in server code creates a bug (or reveals a pre-existing bug) which means that every scheduler contact by a specific class of users fails and results in a ghost.
I have seen two major outbreaks of this class of mass haunting: in May 2007 (SETI only - trac [trac]#194[/trac]), and August 2008 (SETI and Einstein - trac [trac]#713[/trac]). Both instances were tracked down to problems with the Anonymous Platform mechanism.
I would still call the "opposite" scenario a "ghost", as it all boils down to one of the two sides not knowing about the other... Obviously the opposite scenario is worse from a PR standpoint...
Anyway, the point is that there are problems out there that would appear to be far more deserving of the attention that Mr. Anderson puts towards the "parity" issue...
Download limit's. For this I would like to see a system where a host earn's its limit, probably connected to RAC. So that we see less reports as seen in A Feast of Strange Resends??. This would also help with the re-sends problem.
Network Activity. I would like to be able to suspend Network Activity by Project. So that I could stop comms attempts to projects that are down, or in recovery mode, but not effect the scheduler debt calculations which is affected by using the 'no new tasks' or 'suspend' buttons.
Manager quick link buttons. Would like to see all the buttons except for the Project Home Page to be removed. Reason, so that more people would enter by the front door and hopefully read the News.
Bottom of list, Credit Parity.
And before anything else is tried lets see how Eric Korpela's idea pans out. (note to Brian this idea is NOT a Dr.A thing, I thought Eric's answer to you posted 6 Feb 2008 1:39:13 UTC made it clear he didn't like Dr.A's idea's either.)
Download limit's. For this I would like to see a system where a host earn's its limit, probably connected to RAC. So that we see less reports as seen in A Feast of Strange Resends??. This would also help with the re-sends problem.
Agreed. It would also help the cases of people deciding that BOINC was not for them, but not aborting tasks and reporting back the aborts and/or detaching, aka the "participant retention problem"...
Quote:
Network Activity. I would like to be able to suspend Network Activity by Project. So that I could stop comms attempts to projects that are down, or in recovery mode, but not effect the scheduler debt calculations which is affected by using the 'no new tasks' or 'suspend' buttons.
Also a good idea. Right now, I didn't want to pick up additional Einstein work automatically on my AMD because I am wanting to try to see if I can help Cosmology get through the apparent AMD-shortage that their latest changes seem to have caused (a large farm or several large farms of X2 3800+ systems seem to have gone silent), but yet I don't want Cosmology to be allowed automatic retrieval either because who knows what mess may come down. I want to have individual control over both right now, which you can't do except by setting to NNT...
Quote:
Manager quick link buttons. Would like to see all the buttons except for the Project Home Page to be removed. Reason, so that more people would enter by the front door and hopefully read the News.
Disagree. Forcing people to go to the front page and then the account pages will irritate those who liked having the quicker access, and it's likely that those who were not reading the project news to begin with still won't...
Quote:
Bottom of list, Credit Parity.
And before anything else is tried lets see how Eric Korpela's idea pans out. (note to Brian this idea is NOT a Dr.A thing, I thought Eric's answer to you posted 6 Feb 2008 1:39:13 UTC made it clear he didn't like Dr.A's idea's either.)
Disagree on 2 counts, agree that Eric said he didn't like the Jan/Feb idea...
It should have started to become clear to you that there are problems with credit consistency over time. I personally think that inter-project "parity" needs to be given up on, but intra-project parity needs to be improved. When you have a sliding scale of a "median" computer, my computer will likely not get the same credit for the same work over an extended period of time. Sure, other people are equally affected by the sliding scale, but this assumes constant participation by all and has no consideration for systems with discontinuous involvement nor does it consider new systems joining.
I wasn't joking when I said that you (general sense) do not see the deflationary actions of what you are proposing. You are so concerned about fighting the possible inflation fears that you appear to be oblivious to the deflation. It is either that explanation or that since it curtails inflation that you don't care about any deflationary effects.
Finally, David Anderson is still the Director of SETI. Even if the plan was developed completely by Eric, the motivation behind it is at least in part to develop some sort of mechanism to satisfy David Anderson's obsession with this subject. Eric will have to play "go along to get along" to some extent with David. My read of the background information on the current plan indicates that there were political realities that had to be addressed along with a real mathematical issue. I'd say that this likely means that David's ideology played some sort of role in the formulation of the plan.
I did not know that resend_lost_results was broken (this code was written by David Anderson and me). It is expensive to make the necessary database query, and this is simply unavoidable, so some projects can't afford that database strain.
The idea of a transaction-based communication is interesting. At least in principle one could wrap the entire scheduler interaction into a transaction and then try and get a positive acknowledgment from the BOINC client before committing it. I'm not sure if the current design/code would permit this.
I would be interested in understanding better how ghost results are generated.
By the way, looking at the current E@H scheduler log I see:
which means that less than six percent of results are re-sent. And I imagine that this ratio will become smaller as the effects of our server upgrade two weeks ago fades away.
Hi Dr. Allen,
Since you brought up the matter of the scheduler log, is there any reason the link to it for a given host cannot be reactivated?
I know most projects don't have it enabled, but it sure comes in handy when troubleshooting or investigating certain BOINC behaviour.
RE: In the BOINC Survey,
)
I've repeatedly said that it does not appear to be very large of an issue.
Not only that, but there is no guarantee against a "more credits" vote actually being cast by someone who really doesn't feel that way and their agenda was simply to make the figure look bigger.
I'm NOT saying that did happen, but I am saying that there does not appear to have been any mechanism in place to prevent it from happening. The survey was all "honor system", and thus should not be taken as 100% gospel.*
* Yes, I know that also means that people who were/are in it for more credits could've also chose another option to hide their real intent as well...
Also, one of the projects that has recently been highly demonized for giving "too much" was Cosmology. Right now, Cosmology's work rate (and thus credit rate as well) has essentially imploded. Take a look at my productivity there. They took the route of "credits were generous before, so now during this time of problems, nobody gets any credit, so it all evens out". I'm still processing some tasks there, but turnaround time is real slow because Homogeneous Redundancy is enabled there and it appears as though the upheaval completely uprooted a large group of X2 3800+ systems that I was routinely paired up with, so I have tasks that are languishing waiting on another system in my same HR class to come along. On top of that, the project decided to do server-side aborts last week, but not in the "friendly" way (in-progress will still get counted, only non-started get aborted), trapping some tasks in this limbo-land of having 1 success, 1 no reply, 1 "didn't need" or "redundant", and 1 "unsent"...with a quorum of 2 (?!?!?!?!)
RE: Trying to think this
)
No, the issue has been around for quite some time, but it is routinely made out to be more severe of an issue than it really is. The general "fear" goes like this:
*Project X realizes that a certain segment of users, the competitive users, might be more inclined to participate in their project if they are offered more credit than what Project Y offers.
*Project X increases credit.
*The competitive users flock to Project X, leaving Project Y to figure out whether or not they have enough remaining participants to continue operating within their desired parameters.
*Project Y may then decide that they do not have enough participants, so, learing from Project X's example, Project Y increases their credit rate to more than Project X's credit rate.
*Credit war ensues.
*Projects A-W and Z notice the war and decide that they need more participants too, so they join in and thus you have ever-escalating credit rates.
This scenario, while possible, has just never materialized.
I do not know why this is of such great importance to Mr. Anderson, given that it hasn't happened, and doesn't appear likely to happen given that the current project admins over the past few years have not even responded to the fact that the chart at BOINC Combined Statistics showed them lower than SETI and Einstein and they still did not take action to change that...
The whole thing is akin to "tilting at windmills", in my opinion...
@Bruce: *The
)
@Bruce:
*Give users the ability to abort tasks via the project web pages. This goes hand-in-hand with the ghost result problem as well as gives a user remote-cancellation capabilities in the event of being away from the host(s) in question.
RE: If you remove the idea
)
'... therefore surely the next step ...' is an incorrect linkage without logical basis. We are not discussing credit abolition, and I don't know anyone who has proposed that. Credit could be abolished tomorrow, with the attendant problems you mention, regardless of how the cross-project issue is resolved.
People can apply their own cross-project metrics to suit whatever criteria can be imagined, without bounds! Thus they can avoid the trials of continually having to seek validation in various forums, and those who don't have that focus need not respond. It is precisely this type of angst generated by the cross project credit issue that ought to be eliminated. Perhaps the time has come for project credit policies to be dominated by their respective loyal contributors, and not by those who may or not be there next week depending upon the price of some perpetual negotiation. Personally I think it is time the bluff was called on this negative type of competitive behaviour. I say negative because it retards, not advances, the underlying science efforts through diversion of energies. E@H ( like many projects ) is definitely a very co-operative venture of quite significant scientific importance, and ought not be hampered by those whose competitive energies would be better discharged elsewhere if they are incapable of subjugating themselves to common causes.
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
Hi Brian, RE: *The
)
Hi Brian,
I did not know that resend_lost_results was broken (this code was written by David Anderson and me). It is expensive to make the necessary database query, and this is simply unavoidable, so some projects can't afford that database strain.
The idea of a transaction-based communication is interesting. At least in principle one could wrap the entire scheduler interaction into a transaction and then try and get a positive acknowledgment from the BOINC client before committing it. I'm not sure if the current design/code would permit this.
I would be interested in understanding better how ghost results are generated.
By the way, looking at the current E@H scheduler log I see:
[pre]
[root@einstein log_einstein]# grep 2008-08-15 scheduler.log | grep 'Sending reply' | wc
54927 1006554 8047104
[root@einstein log_einstein]# grep 2008-08-15 scheduler.log | grep 'resent' | wc
3290 37331 295560
[/pre]
which means that less than six percent of results are re-sent. And I imagine that this ratio will become smaller as the effects of our server upgrade two weeks ago fades away.
Director, Einstein@Home
RE: I did not know that
)
I wouldn't say that resend_lost_results was broken. It works well at SETI Beta, and it has worked well here except for the brief interlude at the beginning of August when it was re-sending results than couldn't be processed (now fixed).
There is an ongoing problem at SETI where resend_lost_results has been deliberately switched off by the project administrators because they found that the database query load was unsustainable in their particular environment.
In my experience, there are two ways that 'ghost results' come to be generated:
1) Sporadically, when a communications problem means that a client-scheduler interaction doesn't come to a clean conclusion. Specifically, the server scheduler allocates a task to the host and updates the WU and task records accordingly: but the message telling the client about the task is not received, typically because of an http error or timeout - most common at times of high network traffic (SETI: outage recovery. Einstein: old run cleanup/new run initialisation). Transaction processing would help here: or maybe if the WU/tasks tables were not updated to reflect the task allocation until after the client acknowledges receipt of the task allocation message. Of course, that could cause the opposite problem (what's the opposite of a ghost?): the client receives the task instructions and starts processing it, but the server fails to receive the acknowledgement and thinks the allocation has failed. Cue a surprise when the results of the completed task are reported.
2) Systemically, when a change in server code creates a bug (or reveals a pre-existing bug) which means that every scheduler contact by a specific class of users fails and results in a ghost.
I have seen two major outbreaks of this class of mass haunting: in May 2007 (SETI only - trac [trac]#194[/trac]), and August 2008 (SETI and Einstein - trac [trac]#713[/trac]). Both instances were tracked down to problems with the Anonymous Platform mechanism.
RE: RE: I did not know
)
Cosmology also switched it off due to the high number of tasks in people's caches. Since it has to build such a large query list, I think any project that ends up with a fairly large result table to have to look through would end up having similar problems... Due to the 3-5X increase in processing time at Cosmology, I would imagine that they could turn the functionality back on now... They have had several instances where people are stating that they see results from the web page that are not on their local machine(s), thus they have had "Class 5 full roaming vapors" (aka "ghosts")
I would still call the "opposite" scenario a "ghost", as it all boils down to one of the two sides not knowing about the other... Obviously the opposite scenario is worse from a PR standpoint...
Anyway, the point is that there are problems out there that would appear to be far more deserving of the attention that Mr. Anderson puts towards the "parity" issue...
BOINC issue's I would like to
)
BOINC issue's I would like to see improved are:
Download limit's. For this I would like to see a system where a host earn's its limit, probably connected to RAC. So that we see less reports as seen in A Feast of Strange Resends??. This would also help with the re-sends problem.
Network Activity. I would like to be able to suspend Network Activity by Project. So that I could stop comms attempts to projects that are down, or in recovery mode, but not effect the scheduler debt calculations which is affected by using the 'no new tasks' or 'suspend' buttons.
Manager quick link buttons. Would like to see all the buttons except for the Project Home Page to be removed. Reason, so that more people would enter by the front door and hopefully read the News.
Bottom of list, Credit Parity.
And before anything else is tried lets see how Eric Korpela's idea pans out. (note to Brian this idea is NOT a Dr.A thing, I thought Eric's answer to you posted 6 Feb 2008 1:39:13 UTC made it clear he didn't like Dr.A's idea's either.)
RE: BOINC issue's I would
)
Agreed. It would also help the cases of people deciding that BOINC was not for them, but not aborting tasks and reporting back the aborts and/or detaching, aka the "participant retention problem"...
Also a good idea. Right now, I didn't want to pick up additional Einstein work automatically on my AMD because I am wanting to try to see if I can help Cosmology get through the apparent AMD-shortage that their latest changes seem to have caused (a large farm or several large farms of X2 3800+ systems seem to have gone silent), but yet I don't want Cosmology to be allowed automatic retrieval either because who knows what mess may come down. I want to have individual control over both right now, which you can't do except by setting to NNT...
Disagree. Forcing people to go to the front page and then the account pages will irritate those who liked having the quicker access, and it's likely that those who were not reading the project news to begin with still won't...
Disagree on 2 counts, agree that Eric said he didn't like the Jan/Feb idea...
It should have started to become clear to you that there are problems with credit consistency over time. I personally think that inter-project "parity" needs to be given up on, but intra-project parity needs to be improved. When you have a sliding scale of a "median" computer, my computer will likely not get the same credit for the same work over an extended period of time. Sure, other people are equally affected by the sliding scale, but this assumes constant participation by all and has no consideration for systems with discontinuous involvement nor does it consider new systems joining.
I wasn't joking when I said that you (general sense) do not see the deflationary actions of what you are proposing. You are so concerned about fighting the possible inflation fears that you appear to be oblivious to the deflation. It is either that explanation or that since it curtails inflation that you don't care about any deflationary effects.
Finally, David Anderson is still the Director of SETI. Even if the plan was developed completely by Eric, the motivation behind it is at least in part to develop some sort of mechanism to satisfy David Anderson's obsession with this subject. Eric will have to play "go along to get along" to some extent with David. My read of the background information on the current plan indicates that there were political realities that had to be addressed along with a real mathematical issue. I'd say that this likely means that David's ideology played some sort of role in the formulation of the plan.
RE: I did not know that
)
Hi Dr. Allen,
Since you brought up the matter of the scheduler log, is there any reason the link to it for a given host cannot be reactivated?
I know most projects don't have it enabled, but it sure comes in handy when troubleshooting or investigating certain BOINC behaviour.
Alinator