That's exactly what I'm saying. A 10% allocation across 14 days is a maximum of 33.6 hours of work with 100% utilization. It is actually lower than that, but that's getting very technical. There was a HUGE difference in the runtime of S4/S5R1 vs. S5R2/S5R3. You could probably get 4-8 S4/S5R1 results to work on at that low of an allocation and still complete them without entering EDF. However, since you cannot download just part of a result here, the one result is estimating 61 hours, which at 10% simply cannot be done (61 > 33.6).
Yes, we are in agreement on this. But furthermore, I'm saying that in order to get the work done, increase the deadline. Plus I'm saying that I don't want to monitor these message boards and adjust my E@H allocation just because there is a deadline bug in the software/configuration. So I think ultimately we are in agreement.
Quote:
This was all covered in the complaining about S5R2. With your relatively low credit and RAC, I'm going to guess you missed most/all of S5R2. I was a loud complainer back then.
I was a quiet complainer. I set my E@H allocation to zero. I'd rather not have to do so again. And for the record, I'm not chasing credit on this or any other project.
Quote:
You're a "doubting Thomas" (although your name is Bob, I guess). :sigh:
Yes to Bob, but no, after more years of software development than I'd care to admit, just a strong believer in O'Toole's law: Murphy was an optimist.
So basically if i understand the OP. This is my analogy
Einstein set @ 1% share
SETI @ 99% share
A standard pentium 4 @ 3ghz wont finish an einstein unit at 1% within 2 weeks. Therfore it is taking advantage of the EDF system.
Depending on various factors (benchmark, time system is on, time BOINC is running, efficiency), that situation could end up with a message that you just can't do that. As an example, this is what happened with LHC on my system:
Quote:
2007-10-31 18:11:36 [lhcathome] Requesting 2282 seconds of new work
2007-10-31 18:11:41 [lhcathome] Scheduler RPC succeeded [server version 505]
2007-10-31 18:11:41 [lhcathome] Message from server: No work sent
2007-10-31 18:11:41 [lhcathome] Message from server: (won't finish in time) Computer on 98.1% of time, BOINC on 100.0% of that, this project gets 1.0% of that
2007-10-31 18:11:41 [lhcathome] Deferring communication for 1 days 21 hr 1 min 10 sec
2007-10-31 18:11:41 [lhcathome] Reason: requested by project
The deadlines at LHC are around 6 days. 1% of 6 days is only 1728 seconds. My computer wanted 2282 seconds of work. Between it and the LHC servers, I was told "nope, you can't do that".
Why this didn't happen to Bob's system is unknown. What's under the covers is a debt system. If you don't tinker with it, it will look at the debt situation and decide what project should be worked on next. Perhaps Einstein had sufficient debt that needed to be paid back. With his computers being hidden, I can't go look at the history of the machine to make a better guess...
In any case, it will all work out if viewed over a period of time longer than 2 weeks.
I'm saying that in order to get the work done, increase the deadline.
To what? To what works for you and your particular settings?
There comes a point where we as volunteers must be responsible with our settings. I just recently commented on how I have LHC set too low. The only reason I get work from there is because I "cheat" the whole debt system and suspend SETI or Einstein so that it thinks that my "1% allocation" is actually 34% or 67%, depending on the project I suspend. That message I posted where I got rejected by LHC was when I was away from my system and had all 3 projects unsuspended. I need to decide on a real and proper setting for LHC. I joined LHC during the largest SETI outage in the project's history, so the 1% was designed to be a "if neither SETI nor Einstein has work, get some LHC work".
Quote:
So I think ultimately we are in agreement.
We can agree that the deadlines for both S5R2 and S5R3 are too short. I think we would differ on the severity of the S5R3 shortness, with me stating that they are 2-4 days too short and you probably stating that they are 2-3 weeks too short. Even 4 days extra would not get rid of the EDF for you. Like I said though, EDF is only a "problem" if viewed myopically. If you look at several months of runtime, you'll find that it honors your percentages.
Quote:
Quote:
You're a "doubting Thomas" (although your name is Bob, I guess). :sigh:
Yes to Bob, but no, after more years of software development than I'd care to admit, just a strong believer in O'Toole's law: Murphy was an optimist.
I joined LHC during the largest SETI outage in the project's history, so the 1% was designed to be a "if neither SETI nor Einstein has work, get some LHC work".
DOH! It actually wasn't a SETI outage, but an outage here. I knew something was down when I joined LHC. I just got the two events mixed up...
It feels like Einstein is not playing "fair." Is there some sort of adjustment I can make to remedy this? I’d rather not have to drop out in order to restore balance.
Bob
I'm sorry, your logic on this totally escapes me. You set the resource share. Did you take into consideration the longer run times and shorter deadlines with respect to Einstein when you set it up? Would it have been better for Boinc to let the Einstein WU die by strictly following your specific resource share? Then you could have returned a dead WU and not received any credit for doing 61 hours of work.
That feature is there as a safety net for people running more than one project so they can adjust their resource share over time to take into account the different run times and deadlines for different projects without returning dead WU's.
Extending the deadline will not make any difference. People will continue to overextend their resource share no matter how long they have to return a WU. If Einstein made the deadline 6 weeks, there would be someone who could not return the WU on time because their resource share was out of balance.
Extending the deadline will not make any difference. People will continue to overextend their resource share no matter how long they have to return a WU.
Actually, extending deadlines will "help". It will make it to where EDF will never happen, if they are extended out far enough. SETI has done this, but it has the effect of significantly driving up pending credits for people with faster machines and/or realistic resource allocations. The SETI team essentially gave in to the whinging of people who refused to accept that:
Day 1 - CPU allocated 45% Project 1, 5% Project 2, 50% Project 3
Day 2 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 3 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 4 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 5 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 6 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
is identical to
Day 1 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 2 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 3 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 4 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 5 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 6 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
That 0% is what's going to happen with Bob's system as far as Einstein, until such time as debts are repaid to SETI and Rosetta (unless it is tinkered with). There are obviously other combinations of percentages over time. As I have said, the "problem" (and the associated complaints) happen when looking at it with a very short time horizon...
Edit: The deadline extensions probably just end up shoving the EDF "unfair" complaints to other projects. Thus, really, SETI is the one that's not being "fair". The SETI deadlines are now obscenely lengthy, IMO. Bob should take a look at the SETI boards a little more and look at the increase in complaints about pending credit. They are directly related to each other. The increase in deadlines CAUSED the increase in pending.
Finally, Bob, I get what you're saying, and I do fault the staff here for some poor deadline decisions, but the only way you can comfortably stay at a 10% allocation right now and not see ANY incidence of EDF, assuming that the 61 hours is accurate, is for the deadlines here to be set to 4 weeks. Perhaps SSE optimization will help your system enough to stay at 10% and still make 2 weeks. Keep in mind though, as Ned said, "EDF is your friend". Even more so on slower systems...
the "problem" (and the associated complaints) happen when looking at it with a very short time horizon...
Those are the two key points, realistic resource allocation, and, looking at the "problem" in the short term! Exactly!!!!
It will work itself out over the long term if left alone, or, the resource share can be adjusted manually to take into account differences in project run times and deadlines without the problem of going over a deadline and returning a dead WU. Dead WU's are a waste of time and effort all the way around.
My problem with increasing deadlines is that people will look at it and go "Wow, Einstein just increase the amount of time to do a WU" and turn around and decrease the resource share so another project will have more time to run. At that point you are right back where you started. On the other hand, if run times per WU take a drastic increase, then the deadlines should be increased to take that into account.
It will work itself out over the long term if left alone, or, the resource share can be adjusted manually to take into account differences in project run times and deadlines without the problem of going over a deadline and returning a dead WU. Dead WU's are a waste of time and effort all the way around.
Yes, they are. There is though a culture that exists, particularly with SETI, and perhaps specifically with David Anderson, that believes in "set and forget". That's what Bob seems to want to do, set the allocations and forget them. That's fine, so long as he also "forgets" to watch the day-to-day operation. It appears that he still wants to micro-manage. That's fine. I micro-manage this system (my AMD). I have an Intel system that is 100% on SETI that is a service installation. I only look at that system if it goes for more than about 16 hours without reporting. I have periodic issues with my router fouling up my internet connection. I have LAN, but no WAN. A reboot of the router corrects that problem, but sometimes that system has gone into a real long comm backoff mode, so I'll go give it a nudge. 5.8.16 has not been as bad as 5.4.11 was. With BOINC 5.4.11, sometimes it would back off for over a week!
Quote:
if run times per WU take a drastic increase, then the deadlines should be increased to take that into account.
I agree, and deadlines should've been better managed by the project, but if Bob had 10% allocation with one of those monster S5R2 results that would've estimated 120 hours (extrapolating 2X current estimate), that would mean that it would need at least 1,200 hours of wall clock time to complete, which is just over 7 weeks (50 days).
A couple of things would help. First, optimization. Second, only sending the shortest running tasks to slow computers. I forget what they call that, but Einstein used to do it in the S4 days, I think...
In an incredible moment of boredom, and speaking of tinkering and micro-managing, I decided to figure out exactly how much work I have on this system per the current completion time estimates. I supposedly have a 3-day cache. Adding up all the completion estimates, I arrive at about 99.33 hours of work, which is 4.138 days. Somehow, through pure luck probably, I generally maintain my own resource shares (viewed as credit) approximately equal to what I had established, 66/33/1. This link is to this host's stats at BOINCstats. Over the past month, the credit went 64.4% to SETI, 33.0% to Einstein, and 2.6% to LHC. As I said, my allocation of 1% to LHC is too low and I need to adjust it. I have higher pending levels at SETI than I used to have, so that's why it's a smidge low.
Bob, I *know* what asking for more work than what you can hope to complete on time will do. Even with my tinkering, I'm still in no danger of missing any deadline. I always make sure I don't miss. Why do I tinker? Mostly to try to fend off people in my join dates. My system isn't fast enough anymore, so I'm becoming more easily overtaken by groups that have been lingering just behind me for quite some time, as they are upgrading processors or adding hosts. Within 2 months, I'll drop from #13 in my SETI join date to #16, and from #13 in Einstein to #15. I thought about upgrading my processor on this system (the AMD) to an Opteron 175, but I just can't justify the expense, given that I'm only getting unemployment income right now...
I think that some how we are forgetting that all BOINC projects are meant to use spare 'unused' cpu resources when the computer is switched on for normal operation. Nobody should be expecting 24/7 operation. And also it is expected that users be attached to more than one project.
So we shouldn't expect a computer to be able to crunch more than about 20hrs per week for BOINC projects total. If the computer is attached to two projects, one of which is Einstein then a unit cannot be expected to take more than 20hrs to complete.
As observed most users don't like being in EDF mode, so it would appear that Einstein only wants the most modern computers to be attached. As even an AMD 185 hostid=887005 is taking 15+ hrs to complete a unit.
Therefore it is my view that deadlines are too short, especially as the applications have not been fully developed before release.
RE: That's exactly what I'm
)
Yes, we are in agreement on this. But furthermore, I'm saying that in order to get the work done, increase the deadline. Plus I'm saying that I don't want to monitor these message boards and adjust my E@H allocation just because there is a deadline bug in the software/configuration. So I think ultimately we are in agreement.
I was a quiet complainer. I set my E@H allocation to zero. I'd rather not have to do so again. And for the record, I'm not chasing credit on this or any other project.
Yes to Bob, but no, after more years of software development than I'd care to admit, just a strong believer in O'Toole's law: Murphy was an optimist.
RE: So basically if i
)
Depending on various factors (benchmark, time system is on, time BOINC is running, efficiency), that situation could end up with a message that you just can't do that. As an example, this is what happened with LHC on my system:
The deadlines at LHC are around 6 days. 1% of 6 days is only 1728 seconds. My computer wanted 2282 seconds of work. Between it and the LHC servers, I was told "nope, you can't do that".
Why this didn't happen to Bob's system is unknown. What's under the covers is a debt system. If you don't tinker with it, it will look at the debt situation and decide what project should be worked on next. Perhaps Einstein had sufficient debt that needed to be paid back. With his computers being hidden, I can't go look at the history of the machine to make a better guess...
In any case, it will all work out if viewed over a period of time longer than 2 weeks.
RE: I'm saying that in
)
To what? To what works for you and your particular settings?
There comes a point where we as volunteers must be responsible with our settings. I just recently commented on how I have LHC set too low. The only reason I get work from there is because I "cheat" the whole debt system and suspend SETI or Einstein so that it thinks that my "1% allocation" is actually 34% or 67%, depending on the project I suspend. That message I posted where I got rejected by LHC was when I was away from my system and had all 3 projects unsuspended. I need to decide on a real and proper setting for LHC. I joined LHC during the largest SETI outage in the project's history, so the 1% was designed to be a "if neither SETI nor Einstein has work, get some LHC work".
We can agree that the deadlines for both S5R2 and S5R3 are too short. I think we would differ on the severity of the S5R3 shortness, with me stating that they are 2-4 days too short and you probably stating that they are 2-3 weeks too short. Even 4 days extra would not get rid of the EDF for you. Like I said though, EDF is only a "problem" if viewed myopically. If you look at several months of runtime, you'll find that it honors your percentages.
I understand. However, it does work...
Brian
RE: I joined LHC during the
)
DOH! It actually wasn't a SETI outage, but an outage here. I knew something was down when I joined LHC. I just got the two events mixed up...
RE: It feels like
)
I'm sorry, your logic on this totally escapes me. You set the resource share. Did you take into consideration the longer run times and shorter deadlines with respect to Einstein when you set it up? Would it have been better for Boinc to let the Einstein WU die by strictly following your specific resource share? Then you could have returned a dead WU and not received any credit for doing 61 hours of work.
That feature is there as a safety net for people running more than one project so they can adjust their resource share over time to take into account the different run times and deadlines for different projects without returning dead WU's.
Extending the deadline will not make any difference. People will continue to overextend their resource share no matter how long they have to return a WU. If Einstein made the deadline 6 weeks, there would be someone who could not return the WU on time because their resource share was out of balance.
RE: Extending the deadline
)
Actually, extending deadlines will "help". It will make it to where EDF will never happen, if they are extended out far enough. SETI has done this, but it has the effect of significantly driving up pending credits for people with faster machines and/or realistic resource allocations. The SETI team essentially gave in to the whinging of people who refused to accept that:
Day 1 - CPU allocated 45% Project 1, 5% Project 2, 50% Project 3
Day 2 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 3 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 4 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 5 - CPU allocated 88.75% Project 1, 11.25% Project 2, 0% Project 3
Day 6 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
is identical to
Day 1 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 2 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 3 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 4 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 5 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
Day 6 - CPU allocated 80% Project 1, 10% Project 2, 10% Project 3
That 0% is what's going to happen with Bob's system as far as Einstein, until such time as debts are repaid to SETI and Rosetta (unless it is tinkered with). There are obviously other combinations of percentages over time. As I have said, the "problem" (and the associated complaints) happen when looking at it with a very short time horizon...
Edit: The deadline extensions probably just end up shoving the EDF "unfair" complaints to other projects. Thus, really, SETI is the one that's not being "fair". The SETI deadlines are now obscenely lengthy, IMO. Bob should take a look at the SETI boards a little more and look at the increase in complaints about pending credit. They are directly related to each other. The increase in deadlines CAUSED the increase in pending.
Finally, Bob, I get what you're saying, and I do fault the staff here for some poor deadline decisions, but the only way you can comfortably stay at a 10% allocation right now and not see ANY incidence of EDF, assuming that the 61 hours is accurate, is for the deadlines here to be set to 4 weeks. Perhaps SSE optimization will help your system enough to stay at 10% and still make 2 weeks. Keep in mind though, as Ned said, "EDF is your friend". Even more so on slower systems...
Brian
RE: realistic resource
)
Those are the two key points, realistic resource allocation, and, looking at the "problem" in the short term! Exactly!!!!
It will work itself out over the long term if left alone, or, the resource share can be adjusted manually to take into account differences in project run times and deadlines without the problem of going over a deadline and returning a dead WU. Dead WU's are a waste of time and effort all the way around.
My problem with increasing deadlines is that people will look at it and go "Wow, Einstein just increase the amount of time to do a WU" and turn around and decrease the resource share so another project will have more time to run. At that point you are right back where you started. On the other hand, if run times per WU take a drastic increase, then the deadlines should be increased to take that into account.
RE: It will work itself
)
Yes, they are. There is though a culture that exists, particularly with SETI, and perhaps specifically with David Anderson, that believes in "set and forget". That's what Bob seems to want to do, set the allocations and forget them. That's fine, so long as he also "forgets" to watch the day-to-day operation. It appears that he still wants to micro-manage. That's fine. I micro-manage this system (my AMD). I have an Intel system that is 100% on SETI that is a service installation. I only look at that system if it goes for more than about 16 hours without reporting. I have periodic issues with my router fouling up my internet connection. I have LAN, but no WAN. A reboot of the router corrects that problem, but sometimes that system has gone into a real long comm backoff mode, so I'll go give it a nudge. 5.8.16 has not been as bad as 5.4.11 was. With BOINC 5.4.11, sometimes it would back off for over a week!
I agree, and deadlines should've been better managed by the project, but if Bob had 10% allocation with one of those monster S5R2 results that would've estimated 120 hours (extrapolating 2X current estimate), that would mean that it would need at least 1,200 hours of wall clock time to complete, which is just over 7 weeks (50 days).
A couple of things would help. First, optimization. Second, only sending the shortest running tasks to slow computers. I forget what they call that, but Einstein used to do it in the S4 days, I think...
In an incredible moment of
)
In an incredible moment of boredom, and speaking of tinkering and micro-managing, I decided to figure out exactly how much work I have on this system per the current completion time estimates. I supposedly have a 3-day cache. Adding up all the completion estimates, I arrive at about 99.33 hours of work, which is 4.138 days. Somehow, through pure luck probably, I generally maintain my own resource shares (viewed as credit) approximately equal to what I had established, 66/33/1. This link is to this host's stats at BOINCstats. Over the past month, the credit went 64.4% to SETI, 33.0% to Einstein, and 2.6% to LHC. As I said, my allocation of 1% to LHC is too low and I need to adjust it. I have higher pending levels at SETI than I used to have, so that's why it's a smidge low.
Bob, I *know* what asking for more work than what you can hope to complete on time will do. Even with my tinkering, I'm still in no danger of missing any deadline. I always make sure I don't miss. Why do I tinker? Mostly to try to fend off people in my join dates. My system isn't fast enough anymore, so I'm becoming more easily overtaken by groups that have been lingering just behind me for quite some time, as they are upgrading processors or adding hosts. Within 2 months, I'll drop from #13 in my SETI join date to #16, and from #13 in Einstein to #15. I thought about upgrading my processor on this system (the AMD) to an Opteron 175, but I just can't justify the expense, given that I'm only getting unemployment income right now...
Brian
I think that some how we are
)
I think that some how we are forgetting that all BOINC projects are meant to use spare 'unused' cpu resources when the computer is switched on for normal operation. Nobody should be expecting 24/7 operation. And also it is expected that users be attached to more than one project.
So we shouldn't expect a computer to be able to crunch more than about 20hrs per week for BOINC projects total. If the computer is attached to two projects, one of which is Einstein then a unit cannot be expected to take more than 20hrs to complete.
As observed most users don't like being in EDF mode, so it would appear that Einstein only wants the most modern computers to be attached. As even an AMD 185 hostid=887005 is taking 15+ hrs to complete a unit.
Therefore it is my view that deadlines are too short, especially as the applications have not been fully developed before release.