If you have 24/7 operation and always connected Internet than you are probably correct that a low number is preferable.
Yet, I was using a 2 day queue which is very reasonable, and Bruce has been saying that their scheduler is a better predictor of behavior and yet that is not my experience.
Unfortunately, there is no easy way to "prove" much of this as the database is pruned so, um, vigorously that it seems that the results are gone almost before you return them.
I had one WU that I returned just past the deadline by a couple hours and missed the quorum call. Granted this is the rules of the game, however, it appears that the multi-project interaction is falling betwixt the cracks. WOrse, it is disheartening to feel that you just wasted 10 hours of gasoline because of your desire to participate in more than one project.
Anyway, maybe it is my depression running amok again (won't be the last time either if that is what is happening), but we are a community. My personal read, for what that is worth is that we are also one in crisis. I tried to make this point in the developers mailing list and was told that I don't know what I am talking about (not the first time that has happened either).
Yet it is true.
We are a community. Yet sometimes it has the feeling of supplicants pleading on bended knee to the high priests in their ivory towers ... The whole point of BOINC was to be multi-project, so when things like the "perfect storm" we have right now happen, we can keep on trucking.
Yet that does not seem to be the case.
=> SETI@Home is off the air.
=> Einstein@Home like it or not cannot properly schedule if your queue is longer than a day. I have been just getting to the point where I have a 0.1 day queue and it is not clear if that will work better or not.
=> LHC@Home can't seem to track time to complete, based on my data it is a 12% failure rate (goto their forums to read my note there).
=> Predictor@Home is seeming to hold up, though there was a time today when I could not get to the Scheduler. And with the other projects unreliable, or not running on all my systems ... well ...
=> CPDN is my main stay for trying to keep the computers busy. THe problem is that with a queue that is small you only get one WU and that will NOT keep a multi-CPU machine fully employed.
And that is the Crux, even with 5 projects "alive", with a 0.1 day queue ... no ... BOINC is not multi-project capable ...
ANd by the way, the failures of the long run time WUs not being actually being useful is also clearly shown in the fact that my ranking in most projects is steadily falling where prior to this time I was more than holding my own ....
Here Here Mr Buck I concur but not in such strong terms. hehe And I believe Predictor was down because they were loading appliction revision 4.23 and new series t0214 :) Both of which download when you need work Sooo have to rate Predictor right up to CPDN.Hoping when it starts we have graphics. Was in classic CPDN early in classic beta ... wasn't always so smooth... guess they worked out many bugs being stand alone before they joined Boinc
> >
> > Yep, I agree, 0.5 day works well. Most people with deadline problems
> just
> > have their caches too large.
> >
> Maybe it works for you Cochise but for many of us 1-3 days may be required and
> Einstein makes it difficult under the public release of Boinc 4.19. In fact
> upon further review I said ALL other projects I'm in report accurate run times
> to my Benchmarks...welll Predictor actually Overstates my run times by 20-30%
> which is PREFERRED when other projects are accurate. Einstein project
> developers (Bruce I guess) need to be aware of possibly why they are getting
> so many reporting late and this thread is getting many responses and help
> adjust this PLEASE :)
Oh I didnt mean to imply that something couldn't be done to make boinc work better because there is a lot of room for improvement. I just think people are used to having a large cache because they're used to a 2 week deadline and it just doesn't work when the it's only 1 week. BUT, there's a simple fix, lower the cache (for some people anyway). On my machine my average turnaround is 3.5 days so I can't have 2 or 3 work units sitting in the cache. Thats why i have it set to now, .1 I lowered it from .5 days. I have never had a problem connecting and getting more work. But I know that some people are having some real issues and hopefully it will all get worked out soon. I am very pleased that BOINC let's me crunch for 4 projects because there is always something for my machine to work on even when situtations like SETI come up.
> We are a community. Yet sometimes it has the feeling of supplicants pleading
> on bended knee to the high priests in their ivory towers ... The whole point
> of BOINC was to be multi-project, so when things like the "perfect storm" we
> have right now happen, we can keep on trucking.
>
> Yet that does not seem to be the case.
>
> => SETI@Home is off the air.
>
> => Einstein@Home like it or not cannot properly schedule if your queue is
> longer than a day. I have been just getting to the point where I have a 0.1
> day queue and it is not clear if that will work better or not.
>
> => LHC@Home can't seem to track time to complete, based on my data it is a
> 12% failure rate (goto their forums to read my note there).
>
> => Predictor@Home is seeming to hold up, though there was a time today when
> I could not get to the Scheduler. And with the other projects unreliable, or
> not running on all my systems ... well ...
>
> => CPDN is my main stay for trying to keep the computers busy. THe problem
> is that with a queue that is small you only get one WU and that will NOT keep
> a multi-CPU machine fully employed.
Maybe it's just me, but I think the dynamics of multi-project BOINC are a little different that crunching one project.
With just one project, you want to cache days worth of work in case they're down.
When you have five projects, whenever you run out you'll get at least one WU for that project -- that means five WUs in your cache even if your "connect every x days" is damned near zero.
If a project runs out of work, then you'll have four in your cache -- one for each project.
I can confirm that the "Connect to Network every x days" does not cache x Days of Work "in total" (all Projects).
It is rather cacheing x days of work for each Project.
So setting 0.5 days running 5 Projects in parallel results in somewhere around 2.5 days worth of work on that machine.
Right now I'm using something like 0.2 days of work, multiplied by 5 Projects equals my target (~1 day) to achieve Quick Turn-Around times wherever possible.
(since CPDN is one of the Projects, this alone would keep the machine busy at ease anyway, to running out of work is practically impossible)
Actually it's even more, since several Projects also contain "large WorkUnits" that take significantly longer than 0.2 days each.
So 7 days indeed does raise the Limits for both min. System spec and required Resource share in case of multiple Projects, but once Users recognize they are pushing/exceeding the limits, they should quickly learn and adapt to more suitable Settings.
Yes, meeting deadlines can be a problem. Currently there's nothing much else to do than to tinker with queue's, resource share and amount of projects to get a balance in work load and deadlines, if your hosts run 24/7 - if your hosts doesn't run 24/7 it becomes much harder.
With Pirates@home I had alot of trouble meeting the deadlines in the beginning - with equal share on all projects. I didn't think it would be so hard, as the time to complete although varied, are short. But the deadline was too close to meet many times, as the client was going through other projects' wu's first. So many times when boinc finally got to the Pirates wu's they were already way past deadline. To combat that, I took into consideration that Pirates@home only sporadically send out work, and when they do there aren't that many wu's - and then I set resource share for Pirates at 99.90% :D Oh, and I detached my non-24/7 host from Pirates ;) At the moment, I've managed get get pretty close to not missing any deadlines at all, although the non-24/7 host occationally misses one.
I agree with much of the worries John and Paul have stated, but my personal "fear" is if strict resource share is put into BOINC (hopefully this is an unessesarry "fear") - as in:
two projects - resource share is 50/50 - one project is having wu's available all the time, the other every other week. BOINC then makes sure that in a longer period of time, say a month, your host has in fact spend 50% of it's CPU time on each project. So far all is good. But then one of the projects go offline for a couple of months. Your host run the other 100% of the time in that period. When the offline project then comes online again, your host will run that project 100% of the time for a period, to make up for the lost time. In a bigger picture this would mean that for all hosts that run the project that was down a couple of months and run other projects as well, all the other projects will suddenly get much less work done. It may be a scenario that will never be possible, but still, with all that can still be much better with the multi-projects capabilities, I can't help but worry that this may be a possibility in the effords of improving those capabilities. (call me stupid if you want ;) )
> Why is the deadline for WU so short.
> I run 3 project on my computer (24/7), Climate (25%), Seti (50%) and Einstein
> (25%)
> When I recieve WU it seem' that you do not find out that Einstein only run's
> with 25%, the result of this is that the short timelimit will run over, and i
> therefore will not get credit for my work.
>
I've got a similar problem...
since seti has some problem at the moment I added einstein (both at 50%) to have enough work.
my first computer runs a few hours a day and the other about 100h a week (while I'm working).
there is always just 1 seti and 1 einstein WU but it is still hard to keep the time limit for einstein on BOTH computers.
from my point of view the einstein WUs are to big (~30h+ on my computer) an the time limit to small.
> We are a community. Yet sometimes it has the feeling of supplicants pleading
> on bended knee to the high priests in their ivory towers ... The whole point
> of BOINC was to be multi-project, so when things like the "perfect storm" we
> have right now happen, we can keep on trucking.
>
> Yet that does not seem to be the case.
>
> => SETI@Home is off the air.
>
> => Einstein@Home like it or not cannot properly schedule if your queue is
> longer than a day. I have been just getting to the point where I have a 0.1
> day queue and it is not clear if that will work better or not.
>
> => LHC@Home can't seem to track time to complete, based on my data it is a
> 12% failure rate (goto their forums to read my note there).
>
> => Predictor@Home is seeming to hold up, though there was a time today when
> I could not get to the Scheduler. And with the other projects unreliable, or
> not running on all my systems ... well ...
>
> => CPDN is my main stay for trying to keep the computers busy. THe problem
> is that with a queue that is small you only get one WU and that will NOT keep
> a multi-CPU machine fully employed.
>
>
> And that is the Crux, even with 5 projects "alive", with a 0.1 day queue ...
> no ... BOINC is not multi-project capable ...
I agree with you Paul, but I would add that while CPDN may be the most stable of the projects, it is not an option for slower machines (below 800mhz). Given my mix of dial-up and high speed connections, the 0.5 day queue that I have to use for Einstein on the high speed connection has left other projects on my dial-up boxes idle far more than I like (of course SETI being down has only made things worse).
This problem is definatly known to the BOINC developers. There have been several discusions about it on the developers mailing list. I expect (hope) it to be the next thing worked on after the current development version is released, right now getting that out is priority 1.
The current short term scheduler attempts to divide CPU time according to the resource shares in a day. All projects start at zero at the begining of the day and do not remember if they were the only project the day before or anything like that.
The older long term scheduler attempted to divide CPU time over about 1 to 2 weeks. It did this by downloading work from projects that were behind, which would then be crunched starting with the earliest deadline the only time it changed projects was between workunits.
Currently the best looking idea is to try to combine the 2 schedulers. The short term scheduler would still mostly work the same way but instead of directly getting more work it would pass the request to the long term scheduler. The long term scheduler would keep the queue down to a managable length and remember which projects were worked on recently and try to work on other projects next.
There are logic problems with all 3 of these approaches though. The one with the short term scheduler is obvious from this thread, you cannot attach to an unlimited number of projects. The long term scheduler had 2 major problems. With the right combination of queue size, work speed and deadline a project could tie up a computer indefinatly. The other was a long workunit project would be the only thing on the computer for extended periods (2 workunits before a project change). The combination scheduler should eliminate the problem with a project locking the other projects out. It should make it possible to attach to unlimited projects but it still may miss deadlines sometimes depending on how many projects manage to squeeze work in at one time. But only reduce the effect of long workunit projects being the only thing on the computer.
If you have 24/7 operation
)
If you have 24/7 operation and always connected Internet than you are probably correct that a low number is preferable.
Yet, I was using a 2 day queue which is very reasonable, and Bruce has been saying that their scheduler is a better predictor of behavior and yet that is not my experience.
Unfortunately, there is no easy way to "prove" much of this as the database is pruned so, um, vigorously that it seems that the results are gone almost before you return them.
I had one WU that I returned just past the deadline by a couple hours and missed the quorum call. Granted this is the rules of the game, however, it appears that the multi-project interaction is falling betwixt the cracks. WOrse, it is disheartening to feel that you just wasted 10 hours of gasoline because of your desire to participate in more than one project.
Anyway, maybe it is my depression running amok again (won't be the last time either if that is what is happening), but we are a community. My personal read, for what that is worth is that we are also one in crisis. I tried to make this point in the developers mailing list and was told that I don't know what I am talking about (not the first time that has happened either).
Yet it is true.
We are a community. Yet sometimes it has the feeling of supplicants pleading on bended knee to the high priests in their ivory towers ... The whole point of BOINC was to be multi-project, so when things like the "perfect storm" we have right now happen, we can keep on trucking.
Yet that does not seem to be the case.
=> SETI@Home is off the air.
=> Einstein@Home like it or not cannot properly schedule if your queue is longer than a day. I have been just getting to the point where I have a 0.1 day queue and it is not clear if that will work better or not.
=> LHC@Home can't seem to track time to complete, based on my data it is a 12% failure rate (goto their forums to read my note there).
=> Predictor@Home is seeming to hold up, though there was a time today when I could not get to the Scheduler. And with the other projects unreliable, or not running on all my systems ... well ...
=> CPDN is my main stay for trying to keep the computers busy. THe problem is that with a queue that is small you only get one WU and that will NOT keep a multi-CPU machine fully employed.
And that is the Crux, even with 5 projects "alive", with a 0.1 day queue ... no ... BOINC is not multi-project capable ...
ANd by the way, the failures of the long run time WUs not being actually being useful is also clearly shown in the fact that my ranking in most projects is steadily falling where prior to this time I was more than holding my own ....
Here Here Mr Buck I concur
)
Here Here Mr Buck I concur but not in such strong terms. hehe And I believe Predictor was down because they were loading appliction revision 4.23 and new series t0214 :) Both of which download when you need work Sooo have to rate Predictor right up to CPDN.Hoping when it starts we have graphics. Was in classic CPDN early in classic beta ... wasn't always so smooth... guess they worked out many bugs being stand alone before they joined Boinc
> > > > Yep, I agree, 0.5
)
> >
> > Yep, I agree, 0.5 day works well. Most people with deadline problems
> just
> > have their caches too large.
> >
> Maybe it works for you Cochise but for many of us 1-3 days may be required and
> Einstein makes it difficult under the public release of Boinc 4.19. In fact
> upon further review I said ALL other projects I'm in report accurate run times
> to my Benchmarks...welll Predictor actually Overstates my run times by 20-30%
> which is PREFERRED when other projects are accurate. Einstein project
> developers (Bruce I guess) need to be aware of possibly why they are getting
> so many reporting late and this thread is getting many responses and help
> adjust this PLEASE :)
Oh I didnt mean to imply that something couldn't be done to make boinc work better because there is a lot of room for improvement. I just think people are used to having a large cache because they're used to a 2 week deadline and it just doesn't work when the it's only 1 week. BUT, there's a simple fix, lower the cache (for some people anyway). On my machine my average turnaround is 3.5 days so I can't have 2 or 3 work units sitting in the cache. Thats why i have it set to now, .1 I lowered it from .5 days. I have never had a problem connecting and getting more work. But I know that some people are having some real issues and hopefully it will all get worked out soon. I am very pleased that BOINC let's me crunch for 4 projects because there is always something for my machine to work on even when situtations like SETI come up.
Ooopsie.
)
Ooopsie.
> We are a community. Yet
)
> We are a community. Yet sometimes it has the feeling of supplicants pleading
> on bended knee to the high priests in their ivory towers ... The whole point
> of BOINC was to be multi-project, so when things like the "perfect storm" we
> have right now happen, we can keep on trucking.
>
> Yet that does not seem to be the case.
>
> => SETI@Home is off the air.
>
> => Einstein@Home like it or not cannot properly schedule if your queue is
> longer than a day. I have been just getting to the point where I have a 0.1
> day queue and it is not clear if that will work better or not.
>
> => LHC@Home can't seem to track time to complete, based on my data it is a
> 12% failure rate (goto their forums to read my note there).
>
> => Predictor@Home is seeming to hold up, though there was a time today when
> I could not get to the Scheduler. And with the other projects unreliable, or
> not running on all my systems ... well ...
>
> => CPDN is my main stay for trying to keep the computers busy. THe problem
> is that with a queue that is small you only get one WU and that will NOT keep
> a multi-CPU machine fully employed.
Maybe it's just me, but I think the dynamics of multi-project BOINC are a little different that crunching one project.
With just one project, you want to cache days worth of work in case they're down.
When you have five projects, whenever you run out you'll get at least one WU for that project -- that means five WUs in your cache even if your "connect every x days" is damned near zero.
If a project runs out of work, then you'll have four in your cache -- one for each project.
I can confirm that the
)
I can confirm that the "Connect to Network every x days" does not cache x Days of Work "in total" (all Projects).
It is rather cacheing x days of work for each Project.
So setting 0.5 days running 5 Projects in parallel results in somewhere around 2.5 days worth of work on that machine.
Right now I'm using something like 0.2 days of work, multiplied by 5 Projects equals my target (~1 day) to achieve Quick Turn-Around times wherever possible.
(since CPDN is one of the Projects, this alone would keep the machine busy at ease anyway, to running out of work is practically impossible)
Actually it's even more, since several Projects also contain "large WorkUnits" that take significantly longer than 0.2 days each.
So 7 days indeed does raise the Limits for both min. System spec and required Resource share in case of multiple Projects, but once Users recognize they are pushing/exceeding the limits, they should quickly learn and adapt to more suitable Settings.
Yes, meeting deadlines can be
)
Yes, meeting deadlines can be a problem. Currently there's nothing much else to do than to tinker with queue's, resource share and amount of projects to get a balance in work load and deadlines, if your hosts run 24/7 - if your hosts doesn't run 24/7 it becomes much harder.
With Pirates@home I had alot of trouble meeting the deadlines in the beginning - with equal share on all projects. I didn't think it would be so hard, as the time to complete although varied, are short. But the deadline was too close to meet many times, as the client was going through other projects' wu's first. So many times when boinc finally got to the Pirates wu's they were already way past deadline. To combat that, I took into consideration that Pirates@home only sporadically send out work, and when they do there aren't that many wu's - and then I set resource share for Pirates at 99.90% :D Oh, and I detached my non-24/7 host from Pirates ;) At the moment, I've managed get get pretty close to not missing any deadlines at all, although the non-24/7 host occationally misses one.
I agree with much of the worries John and Paul have stated, but my personal "fear" is if strict resource share is put into BOINC (hopefully this is an unessesarry "fear") - as in:
two projects - resource share is 50/50 - one project is having wu's available all the time, the other every other week. BOINC then makes sure that in a longer period of time, say a month, your host has in fact spend 50% of it's CPU time on each project. So far all is good. But then one of the projects go offline for a couple of months. Your host run the other 100% of the time in that period. When the offline project then comes online again, your host will run that project 100% of the time for a period, to make up for the lost time. In a bigger picture this would mean that for all hosts that run the project that was down a couple of months and run other projects as well, all the other projects will suddenly get much less work done. It may be a scenario that will never be possible, but still, with all that can still be much better with the multi-projects capabilities, I can't help but worry that this may be a possibility in the effords of improving those capabilities. (call me stupid if you want ;) )
> Why is the deadline for WU
)
> Why is the deadline for WU so short.
> I run 3 project on my computer (24/7), Climate (25%), Seti (50%) and Einstein
> (25%)
> When I recieve WU it seem' that you do not find out that Einstein only run's
> with 25%, the result of this is that the short timelimit will run over, and i
> therefore will not get credit for my work.
>
I've got a similar problem...
since seti has some problem at the moment I added einstein (both at 50%) to have enough work.
my first computer runs a few hours a day and the other about 100h a week (while I'm working).
there is always just 1 seti and 1 einstein WU but it is still hard to keep the time limit for einstein on BOTH computers.
from my point of view the einstein WUs are to big (~30h+ on my computer) an the time limit to small.
> We are a community. Yet
)
> We are a community. Yet sometimes it has the feeling of supplicants pleading
> on bended knee to the high priests in their ivory towers ... The whole point
> of BOINC was to be multi-project, so when things like the "perfect storm" we
> have right now happen, we can keep on trucking.
>
> Yet that does not seem to be the case.
>
> => SETI@Home is off the air.
>
> => Einstein@Home like it or not cannot properly schedule if your queue is
> longer than a day. I have been just getting to the point where I have a 0.1
> day queue and it is not clear if that will work better or not.
>
> => LHC@Home can't seem to track time to complete, based on my data it is a
> 12% failure rate (goto their forums to read my note there).
>
> => Predictor@Home is seeming to hold up, though there was a time today when
> I could not get to the Scheduler. And with the other projects unreliable, or
> not running on all my systems ... well ...
>
> => CPDN is my main stay for trying to keep the computers busy. THe problem
> is that with a queue that is small you only get one WU and that will NOT keep
> a multi-CPU machine fully employed.
>
>
> And that is the Crux, even with 5 projects "alive", with a 0.1 day queue ...
> no ... BOINC is not multi-project capable ...
I agree with you Paul, but I would add that while CPDN may be the most stable of the projects, it is not an option for slower machines (below 800mhz). Given my mix of dial-up and high speed connections, the 0.5 day queue that I have to use for Einstein on the high speed connection has left other projects on my dial-up boxes idle far more than I like (of course SETI being down has only made things worse).
This problem is definatly
)
This problem is definatly known to the BOINC developers. There have been several discusions about it on the developers mailing list. I expect (hope) it to be the next thing worked on after the current development version is released, right now getting that out is priority 1.
The current short term scheduler attempts to divide CPU time according to the resource shares in a day. All projects start at zero at the begining of the day and do not remember if they were the only project the day before or anything like that.
The older long term scheduler attempted to divide CPU time over about 1 to 2 weeks. It did this by downloading work from projects that were behind, which would then be crunched starting with the earliest deadline the only time it changed projects was between workunits.
Currently the best looking idea is to try to combine the 2 schedulers. The short term scheduler would still mostly work the same way but instead of directly getting more work it would pass the request to the long term scheduler. The long term scheduler would keep the queue down to a managable length and remember which projects were worked on recently and try to work on other projects next.
There are logic problems with all 3 of these approaches though. The one with the short term scheduler is obvious from this thread, you cannot attach to an unlimited number of projects. The long term scheduler had 2 major problems. With the right combination of queue size, work speed and deadline a project could tie up a computer indefinatly. The other was a long workunit project would be the only thing on the computer for extended periods (2 workunits before a project change). The combination scheduler should eliminate the problem with a project locking the other projects out. It should make it possible to attach to unlimited projects but it still may miss deadlines sometimes depending on how many projects manage to squeeze work in at one time. But only reduce the effect of long workunit projects being the only thing on the computer.
BOINC WIKI
BOINCing since 2002/12/8