I found this point quite enlightening, thanks A-D:
> [...] there are two kinds of redundancy to consider:
> 1) how many matching results are required for validation
> 2) how many MORE copies of a work unit than "whatever the answer to (1) is"
> will be sent out to machines
Id split 1 into 1A and 1B:
1A) how many results are required for validation if a project could design its own validation criteria from scratch
1B) how many results are needed for validation if a project chooses to work within the current BOINC system.
2) how many MORE copies of a work unit than "whatever then answer to (1B) is" will be sent out
Whatever you think about the answer to 1A, it is clear that the answer to 1B is 3, as WU are in fact accepted with only 3 computed results. That is not a guess, Randy, that is what you can observe for yourself.
It is also clear that currently the answer to (2) is 4, that is one more than the answer to 1B.
Of course it is right that the project admins make the final choice - nobody suggested hacking into their server and mutinously adjusting the values for ourselves. But when they review that choice, as good admins will regularly review all their ongoing choices, it will be useful for them to know how participants feel.
And how I feel is that no reason has been given to me that satisfies my feeling that the increase from 3 to 4 is a waste of the resources I am donating to this project. The reasons I quoted in my last posting were not guesswork, or at least not guesses on my part; they were points made in other threads in defence of the current strategy. And not one of those reasons makes me stop feeling that 25% on average of the resources I am donating are being wasted.
Knowing that leaves the project admins with a three way choice. They can give a different explanation of why they want to keep things as they are; they can adopt a different strategy (several have been suggested); or they can accept that in the long term I will be donating my resources somewhere else. In the short term I'm happy to stick around and see how things develop.
It would be mindreading to suggest I knew which particular users might also vote with their feet. On a project attracting thousands of participants it is not mind reading to suggest that *some* other participants will feel the same way.
> > Well, OK. Paul says that the probability of errors is "fairly high,"
> and
> > Scott says the extra work unit is "often" needed (much more than a spare
> tire,
> > evidently!) Fine. Obviously, as the percentage of cases in which at
> least
> > one copy of a work unit times out or produces an error approaches 100%,
> it
> > becomes more and more reasonable just to send the work unit to one (or
> more)
> > extra machines in the first place and write off whatever redundancy
> results.
> > This will reduce server strain and reduce the chances that, through sheer
> bad
> > luck, a work unit's progress will be drawn out to ridiculous or
> scientifically
> > inconvenient lengths of time. If we do have a "relative abundance" of
> > computing power (relative to raw data, or to created work units?), and
> > unavoidable financial constraints on server power, then the point at
> which
> > this becomes reasonable will be lower.
> >
> > I don't think I missed any of this. I didn't announce that the present
> system
> > is wrong. I simply questioned whether it is optimal. It isn't,
> strictly
> > speaking, but other factors mean that the project has neither the need
> nor the
> > luxury of reducing redundancy to zero.
> >
>
> If they had to pay for the crunching they would reduce unnecessary redundancy
> in very quick order. I think they have a very blasé attitude to using free
> resources efficiently.
>
RPFT, just what is your agenda? @ minimum you remind me of a union rep trying to extol the virtues of the union against the MAN. REMEMBER you are here of your own free will and may leave at anytime........
A quote from Matt Lebofsky of Seti@Home to support mine/our case that resources are being wasted just so that users can get credit faster and commitment to change this when possible.
Begin Quote:
This came up in conversation the other day. While my points earlier on still stand, one of the main reasons to increase the quorum was to ensure users get credit faster. As we get more users (and therefore there's less waiting-around time), the quorum should drop back down to 3.
We are keenly aware about what resources we have and using them efficiently. For example, check out this page we posted in 2001:
> A quote from Matt Lebofsky of Seti@Home to support mine/our case that
> resources are being wasted just so that users can get credit faster and
> commitment to change this when possible.
Quote snipped. Read it in the previous post if you wish.
This guy claims to have no hidden agenda but in reality you'd have to be a total dickhead to believe that. He is also a distorter of the truth. He takes a lot of trouble to exclude the main post from Matt Lebofsky and show the followup one (which I snipped) only so that he can claim support for his argument.
He started exactly the same thread as this one here over at S@H and got roundly condemned. Matt Lebofsky actually entered that thread at an earlier stage and made the full comment which I've quoted below. You might have thought that RPFT would have been honest enough to refer to that particular comment, as well as the shorter followup. Unfortunately the main comment blows his whole line of reasoning away so....
Anyway, here is what Matt actually said in the full first comment:-
Here's a couple of factoids/things to remember for your redundancy debating pleasure:
1) Depending on the CPU type (sparc, pentium, apple, AMD, etc.) results may slightly. When we compare results to validate them, they must be "fuzzy compared." Normally, the numerical differences are small enough to avoid damaging the science, but large enough that you can't just put an equals sign between two values. Now.. what about the case where, say, the next Pentium causes these values to be out of fuzziness range? Having a fourth result is scientific insurance, no?
2) There are also those who enjoy overclocking their CPUs, which can lead to varying results as well.
3) In Classic SETI@home, we feed out workunits strictly based on demand. At this point, Classic redundancy averages around 6 or 7 (a rough off-hand guess). Depending on database outages, or other crises, we can't generate new workunits but still send out whatever we got. We have had, at rare times, redundancy levels above 20 or 30. We found in general it was better to give people something to do rather than get complaints the system was down and "Berkeley screwed up again, they don't know what they are doing, blah blah blah.." Now with BOINC, we have a cap at 4, and this is a clearly enacted policy (built into the whole system) so nobody can (rightfully) complain about lack of work. Which is worse?
For the record, I was not part of the discussion about raising quorum to 4 so I don't have a strong feeling one way or the other. However, intuitively I feel 4 is a good "sweet spot" when regarding workunit processing times, scientific integrity, and avoiding pointless make-work (while still giving enough work to keep the user base happy).
- Matt
If you read and digest that, 4 is the "sweet spot".
So to RPFT, please take your hidden agenda elsewhere and stop insulting our collective intelligence.
Edit: If any reader wants to see how threadbare RPFT's arguments really are, go over to S@H and see what the collective opinion really was. Quite entertaining actually. Also so predictable that you should realise that it's for amusement value only.
I'm pretty happy with the way it is right now. I get about the same amount of credit that I claim and I complete my 1 to 2 wu's per day with no question about getting work or not. All up, well done team!
> > A quote from Matt Lebofsky of Seti@Home to support mine/our case that
> > resources are being wasted just so that users can get credit faster and
> > commitment to change this when possible.
>
> Quote snipped. Read it in the previous post if you wish.
>
> This guy claims to have no hidden agenda but in reality you'd have to be a
> total dickhead to believe that. He is also a distorter of the truth. He
> takes a lot of trouble to exclude the main post from Matt Lebofsky and show
> the followup one (which I snipped) only so that he can claim support for his
> argument.
>
> He started exactly the same thread as this one here over at S@H and got
> roundly condemned. Matt Lebofsky actually entered that thread at an earlier
> stage and made the full comment which I've quoted below. You might have
> thought that RPFT would have been honest enough to refer to that particular
> comment, as well as the shorter followup. Unfortunately the main comment
> blows his whole line of reasoning away so....
>
> Anyway, here is what Matt actually said in the full first comment:-
>
>
> Here's a couple of factoids/things to remember for your redundancy debating
> pleasure:
>
> 1) Depending on the CPU type (sparc, pentium, apple, AMD, etc.) results may
> slightly. When we compare results to validate them, they must be "fuzzy
> compared." Normally, the numerical differences are small enough to avoid
> damaging the science, but large enough that you can't just put an equals sign
> between two values. Now.. what about the case where, say, the next Pentium
> causes these values to be out of fuzziness range? Having a fourth result is
> scientific insurance, no?
>
> 2) There are also those who enjoy overclocking their CPUs, which can lead to
> varying results as well.
>
> 3) In Classic SETI@home, we feed out workunits strictly based on demand. At
> this point, Classic redundancy averages around 6 or 7 (a rough off-hand
> guess). Depending on database outages, or other crises, we can't generate new
> workunits but still send out whatever we got. We have had, at rare times,
> redundancy levels above 20 or 30. We found in general it was better to give
> people something to do rather than get complaints the system was down and
> "Berkeley screwed up again, they don't know what they are doing, blah blah
> blah.." Now with BOINC, we have a cap at 4, and this is a clearly enacted
> policy (built into the whole system) so nobody can (rightfully) complain about
> lack of work. Which is worse?
>
> For the record, I was not part of the discussion about raising quorum to 4 so
> I don't have a strong feeling one way or the other. However, intuitively I
> feel 4 is a good "sweet spot" when regarding workunit processing times,
> scientific integrity, and avoiding pointless make-work (while still giving
> enough work to keep the user base happy).
>
> - Matt
>
>
> If you read and digest that, 4 is the "sweet spot".
>
> So to RPFT, please take your hidden agenda elsewhere and stop insulting our
> collective intelligence.
>
> Edit: If any reader wants to see how threadbare RPFT's arguments really are,
> go over to S@H and see what the collective opinion really was. Quite
> entertaining actually. Also so predictable that you should realise that it's
> for amusement value only.
>
In the light of the 2nd post the 1st becomes irrelevant;
"one of the main reasons to increase the quorum was to ensure users get credit faster."
RPFT, you've proven to not actually respond to people's points and continue to spout out yours without regard with what others have written, except in the few cases where you can snip here and there and make what someone has said fit into what you believe. As your quote says itself "One of the main reasons". Not -the- main reason. Just one of the main reason, i.e. there were other main reasons as well, which were stated above. But you say those are "irrelevant" now.
As a NUMBER of people have pointed out to you, because of the way BOINC uses resource share (i.e. each machine processes a percentage of it's time for projects as the user has defined, regardless of how fast the processing for each project is), if a project has redundancy built in, it slows down ONLY THAT PROJECT. I'm not entirely certain why you continue to believe differently, as you have refused to respond in either of the two threads to anyone that has pointed this out to you. If I give 12 hours of my computer's time per day to SETI and 12 hours to Einstein, then if SETI is more redundant than Einstein the only project that is hurt is SETI (noting that this still does not "prove" that the redundancy is higher than it needs to be) as Einstein still gets it's 12 hours in.
If your agenda is only to make sure that "no projects suffer" due to redundancy, then a number of people have pointed out to you that the only way a project can suffer from excessive redundancy is for that project to code for it. Project A's redundancy has no effect on Project B's, unless Project A would have run out of work and Project B gotten those computing cycles. However, few of the projects here seem to be at a lose for work to compute.
1. When the message you are responding to is lengthy and your response is just a few words, could you please do us the courtesy of only using enough context to clearly identify the bit you are responding to. If someone wants to read the whole post all over again, then it is right next door and they can easily go back. I thought you were supposedly into eliminating redundancy??
2. If you make a statement that is clearly and demonstrably false, eg the famous "deprive other projects" statement, it would probably be a wise move to face up to it and admit you were wrong, and then move on rather than trying to tough it out with watered down versions like, "if any project wastes part of the pool of resources then all projects will suffer". Clearly all projects don't suffer.
> In the light of the 2nd post the 1st becomes irrelevant;
And that is exactly why Matt said, "While my points earlier on still stand....",
is that what you are trying to have us believe?? A balanced person reading all of what Matt had to say would form the opinion that 4 is about right from the projects' point of view.
> "one of the main reasons to increase the quorum was to ensure users get credit
> faster."
Actually, that reads to me as meaning just one of several reasons, which he clearly enunciated in his first post, which you would have us believe is irrelevant!!
You are really flogging a dead horse on this issue. Please let us all move on!!
I found this point quite
)
I found this point quite enlightening, thanks A-D:
> [...] there are two kinds of redundancy to consider:
> 1) how many matching results are required for validation
> 2) how many MORE copies of a work unit than "whatever the answer to (1) is"
> will be sent out to machines
Id split 1 into 1A and 1B:
1A) how many results are required for validation if a project could design its own validation criteria from scratch
1B) how many results are needed for validation if a project chooses to work within the current BOINC system.
2) how many MORE copies of a work unit than "whatever then answer to (1B) is" will be sent out
Whatever you think about the answer to 1A, it is clear that the answer to 1B is 3, as WU are in fact accepted with only 3 computed results. That is not a guess, Randy, that is what you can observe for yourself.
It is also clear that currently the answer to (2) is 4, that is one more than the answer to 1B.
Of course it is right that the project admins make the final choice - nobody suggested hacking into their server and mutinously adjusting the values for ourselves. But when they review that choice, as good admins will regularly review all their ongoing choices, it will be useful for them to know how participants feel.
And how I feel is that no reason has been given to me that satisfies my feeling that the increase from 3 to 4 is a waste of the resources I am donating to this project. The reasons I quoted in my last posting were not guesswork, or at least not guesses on my part; they were points made in other threads in defence of the current strategy. And not one of those reasons makes me stop feeling that 25% on average of the resources I am donating are being wasted.
Knowing that leaves the project admins with a three way choice. They can give a different explanation of why they want to keep things as they are; they can adopt a different strategy (several have been suggested); or they can accept that in the long term I will be donating my resources somewhere else. In the short term I'm happy to stick around and see how things develop.
It would be mindreading to suggest I knew which particular users might also vote with their feet. On a project attracting thousands of participants it is not mind reading to suggest that *some* other participants will feel the same way.
~~gravywavy
> > Well, OK. Paul says that
)
> > Well, OK. Paul says that the probability of errors is "fairly high,"
> and
> > Scott says the extra work unit is "often" needed (much more than a spare
> tire,
> > evidently!) Fine. Obviously, as the percentage of cases in which at
> least
> > one copy of a work unit times out or produces an error approaches 100%,
> it
> > becomes more and more reasonable just to send the work unit to one (or
> more)
> > extra machines in the first place and write off whatever redundancy
> results.
> > This will reduce server strain and reduce the chances that, through sheer
> bad
> > luck, a work unit's progress will be drawn out to ridiculous or
> scientifically
> > inconvenient lengths of time. If we do have a "relative abundance" of
> > computing power (relative to raw data, or to created work units?), and
> > unavoidable financial constraints on server power, then the point at
> which
> > this becomes reasonable will be lower.
> >
> > I don't think I missed any of this. I didn't announce that the present
> system
> > is wrong. I simply questioned whether it is optimal. It isn't,
> strictly
> > speaking, but other factors mean that the project has neither the need
> nor the
> > luxury of reducing redundancy to zero.
> >
>
> If they had to pay for the crunching they would reduce unnecessary redundancy
> in very quick order. I think they have a very blasé attitude to using free
> resources efficiently.
>
RPFT, just what is your agenda? @ minimum you remind me of a union rep trying to extol the virtues of the union against the MAN. REMEMBER you are here of your own free will and may leave at anytime........
Link to Unofficial Wiki for BOINC, by Paul and Friends
A quote from Matt Lebofsky of
)
A quote from Matt Lebofsky of Seti@Home to support mine/our case that resources are being wasted just so that users can get credit faster and commitment to change this when possible.
Begin Quote:
This came up in conversation the other day. While my points earlier on still stand, one of the main reasons to increase the quorum was to ensure users get credit faster. As we get more users (and therefore there's less waiting-around time), the quorum should drop back down to 3.
We are keenly aware about what resources we have and using them efficiently. For example, check out this page we posted in 2001:
http://setiathome.berkeley.edu/green.html
I mean.. why do you think BOINC got started in the first place?
- Matt
BOINC/SETI@home
End Quote
I have no agenda other than to support all the great BOINC projects that are currently online and those yet to come online.
To that end, if any project wastes part of the pool of resources then all projects will suffer.
> A quote from Matt Lebofsky
)
> A quote from Matt Lebofsky of Seti@Home to support mine/our case that
> resources are being wasted just so that users can get credit faster and
> commitment to change this when possible.
Quote snipped. Read it in the previous post if you wish.
This guy claims to have no hidden agenda but in reality you'd have to be a total dickhead to believe that. He is also a distorter of the truth. He takes a lot of trouble to exclude the main post from Matt Lebofsky and show the followup one (which I snipped) only so that he can claim support for his argument.
He started exactly the same thread as this one here over at S@H and got roundly condemned. Matt Lebofsky actually entered that thread at an earlier stage and made the full comment which I've quoted below. You might have thought that RPFT would have been honest enough to refer to that particular comment, as well as the shorter followup. Unfortunately the main comment blows his whole line of reasoning away so....
Anyway, here is what Matt actually said in the full first comment:-
Here's a couple of factoids/things to remember for your redundancy debating pleasure:
1) Depending on the CPU type (sparc, pentium, apple, AMD, etc.) results may slightly. When we compare results to validate them, they must be "fuzzy compared." Normally, the numerical differences are small enough to avoid damaging the science, but large enough that you can't just put an equals sign between two values. Now.. what about the case where, say, the next Pentium causes these values to be out of fuzziness range? Having a fourth result is scientific insurance, no?
2) There are also those who enjoy overclocking their CPUs, which can lead to varying results as well.
3) In Classic SETI@home, we feed out workunits strictly based on demand. At this point, Classic redundancy averages around 6 or 7 (a rough off-hand guess). Depending on database outages, or other crises, we can't generate new workunits but still send out whatever we got. We have had, at rare times, redundancy levels above 20 or 30. We found in general it was better to give people something to do rather than get complaints the system was down and "Berkeley screwed up again, they don't know what they are doing, blah blah blah.." Now with BOINC, we have a cap at 4, and this is a clearly enacted policy (built into the whole system) so nobody can (rightfully) complain about lack of work. Which is worse?
For the record, I was not part of the discussion about raising quorum to 4 so I don't have a strong feeling one way or the other. However, intuitively I feel 4 is a good "sweet spot" when regarding workunit processing times, scientific integrity, and avoiding pointless make-work (while still giving enough work to keep the user base happy).
- Matt
If you read and digest that, 4 is the "sweet spot".
So to RPFT, please take your hidden agenda elsewhere and stop insulting our collective intelligence.
Edit: If any reader wants to see how threadbare RPFT's arguments really are, go over to S@H and see what the collective opinion really was. Quite entertaining actually. Also so predictable that you should realise that it's for amusement value only.
Cheers,
Gary.
I'm pretty happy with the way
)
I'm pretty happy with the way it is right now. I get about the same amount of credit that I claim and I complete my 1 to 2 wu's per day with no question about getting work or not. All up, well done team!
Live long and crunch!
Paul
> > A quote from Matt
)
> > A quote from Matt Lebofsky of Seti@Home to support mine/our case that
> > resources are being wasted just so that users can get credit faster and
> > commitment to change this when possible.
>
> Quote snipped. Read it in the previous post if you wish.
>
> This guy claims to have no hidden agenda but in reality you'd have to be a
> total dickhead to believe that. He is also a distorter of the truth. He
> takes a lot of trouble to exclude the main post from Matt Lebofsky and show
> the followup one (which I snipped) only so that he can claim support for his
> argument.
>
> He started exactly the same thread as this one here over at S@H and got
> roundly condemned. Matt Lebofsky actually entered that thread at an earlier
> stage and made the full comment which I've quoted below. You might have
> thought that RPFT would have been honest enough to refer to that particular
> comment, as well as the shorter followup. Unfortunately the main comment
> blows his whole line of reasoning away so....
>
> Anyway, here is what Matt actually said in the full first comment:-
>
>
> Here's a couple of factoids/things to remember for your redundancy debating
> pleasure:
>
> 1) Depending on the CPU type (sparc, pentium, apple, AMD, etc.) results may
> slightly. When we compare results to validate them, they must be "fuzzy
> compared." Normally, the numerical differences are small enough to avoid
> damaging the science, but large enough that you can't just put an equals sign
> between two values. Now.. what about the case where, say, the next Pentium
> causes these values to be out of fuzziness range? Having a fourth result is
> scientific insurance, no?
>
> 2) There are also those who enjoy overclocking their CPUs, which can lead to
> varying results as well.
>
> 3) In Classic SETI@home, we feed out workunits strictly based on demand. At
> this point, Classic redundancy averages around 6 or 7 (a rough off-hand
> guess). Depending on database outages, or other crises, we can't generate new
> workunits but still send out whatever we got. We have had, at rare times,
> redundancy levels above 20 or 30. We found in general it was better to give
> people something to do rather than get complaints the system was down and
> "Berkeley screwed up again, they don't know what they are doing, blah blah
> blah.." Now with BOINC, we have a cap at 4, and this is a clearly enacted
> policy (built into the whole system) so nobody can (rightfully) complain about
> lack of work. Which is worse?
>
> For the record, I was not part of the discussion about raising quorum to 4 so
> I don't have a strong feeling one way or the other. However, intuitively I
> feel 4 is a good "sweet spot" when regarding workunit processing times,
> scientific integrity, and avoiding pointless make-work (while still giving
> enough work to keep the user base happy).
>
> - Matt
>
>
> If you read and digest that, 4 is the "sweet spot".
>
> So to RPFT, please take your hidden agenda elsewhere and stop insulting our
> collective intelligence.
>
> Edit: If any reader wants to see how threadbare RPFT's arguments really are,
> go over to S@H and see what the collective opinion really was. Quite
> entertaining actually. Also so predictable that you should realise that it's
> for amusement value only.
>
In the light of the 2nd post the 1st becomes irrelevant;
"one of the main reasons to increase the quorum was to ensure users get credit faster."
RPFT, you've proven to not
)
RPFT, you've proven to not actually respond to people's points and continue to spout out yours without regard with what others have written, except in the few cases where you can snip here and there and make what someone has said fit into what you believe. As your quote says itself "One of the main reasons". Not -the- main reason. Just one of the main reason, i.e. there were other main reasons as well, which were stated above. But you say those are "irrelevant" now.
As a NUMBER of people have pointed out to you, because of the way BOINC uses resource share (i.e. each machine processes a percentage of it's time for projects as the user has defined, regardless of how fast the processing for each project is), if a project has redundancy built in, it slows down ONLY THAT PROJECT. I'm not entirely certain why you continue to believe differently, as you have refused to respond in either of the two threads to anyone that has pointed this out to you. If I give 12 hours of my computer's time per day to SETI and 12 hours to Einstein, then if SETI is more redundant than Einstein the only project that is hurt is SETI (noting that this still does not "prove" that the redundancy is higher than it needs to be) as Einstein still gets it's 12 hours in.
If your agenda is only to make sure that "no projects suffer" due to redundancy, then a number of people have pointed out to you that the only way a project can suffer from excessive redundancy is for that project to code for it. Project A's redundancy has no effect on Project B's, unless Project A would have run out of work and Project B gotten those computing cycles. However, few of the projects here seem to be at a lose for work to compute.
So, what is your real agenda then?
RPFT, A couple of
)
RPFT,
A couple of points:-
1. When the message you are responding to is lengthy and your response is just a few words, could you please do us the courtesy of only using enough context to clearly identify the bit you are responding to. If someone wants to read the whole post all over again, then it is right next door and they can easily go back. I thought you were supposedly into eliminating redundancy??
2. If you make a statement that is clearly and demonstrably false, eg the famous "deprive other projects" statement, it would probably be a wise move to face up to it and admit you were wrong, and then move on rather than trying to tough it out with watered down versions like, "if any project wastes part of the pool of resources then all projects will suffer". Clearly all projects don't suffer.
> In the light of the 2nd post the 1st becomes irrelevant;
And that is exactly why Matt said, "While my points earlier on still stand....",
is that what you are trying to have us believe?? A balanced person reading all of what Matt had to say would form the opinion that 4 is about right from the projects' point of view.
> "one of the main reasons to increase the quorum was to ensure users get credit
> faster."
Actually, that reads to me as meaning just one of several reasons, which he clearly enunciated in his first post, which you would have us believe is irrelevant!!
You are really flogging a dead horse on this issue. Please let us all move on!!
Cheers,
Gary.