BOINC and Einstein particularly don't play by the rules ...

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5875
Credit: 118476111518
RAC: 26010995

RE: ... Where in the

Quote:
... Where in the Einstein forum rules does it lay out what is acceptable or non-acceptable postings.


When you are composing a message, have a look immediately to the left of the message composition box and you will see a set of 6 "rules". #4 says, "No messages intended to annoy or antagonize other people, or to hijack a thread". Your message annoyed me (and probably others) because it was making a quite unjustified complaint against the project. I don't moderate posts just because I'm annoyed. If you had posted the same message in your own thread, I may not have even responded, since trying to reason with someone who is frustrated tends to be unsuccessful. Your thread was moderated because of the last part of the rule.

Quote:
... it would have been courteous of the moderators to at least have left a message in the original thread that I was subscribed to that my posts had been moved to a separate thread. I was surprised to say the least to see my posts deleted without comment.


The standard procedure has a "courteous" component and I used it. When messages are moderated, the reasons why and any comments from the moderator are sent in an email to your registered email address. I took the trouble to include comments as to exactly why the post was moderated. The automated part of the message tells you where the post was moved to. It was NOT "deleted without comment". Have you checked your email?

Quote:
So experts, just what BOINC setting will allow me to download E@H work that won't overcome my system and run me into inevitable deadlines?


I'm not an expert, but I'll make suggestions based on things you've said in various messages. I'll summarise what appear to be at least some of your requirements in the following list. Please correct me if I'm wrong:-

  • * Your preferred project is Seti with a resource share of at least 80%
    * Your other projects are Milkyway and Einstein perhaps with up to 10% each.
    * You would like Einstein to be a backup project to keep your hosts busy if Seti can't supply tasks.
    * You want to have a 4 day cache to handle short term Seti outages but you don't want BOINC to fill up that cache with Einstein.
    * You don't mind having Einstein GPU tasks because they 'pay' more than Milkyway tasks do.

I have no experience with BOINC's backup project mechanism, but it seems to me that the only ways (without continual manual intervention) likely to stop BOINC filling up your 4 day cache with Einstein tasks is either, (a) don't have a 4 day cache, or (b) give Einstein a zero resource share. I'll deal with cache size later.

If you make Einstein a backup project, you probably wouldn't get any Einstein GPU tasks while Milkyway could supply them. So you would really need to make Milkyway a backup project too - if BOINC can have 2 backup projects (I don't know). If it can, and it's been set up correctly, 2 backup projects should share the resources "equally", depending on how BOINC defines "equally". Perhaps the GPU tasks would come alternately from either backup project which wouldn't be terribly "equal". A further complication would be if you were trying to run Einstein GPU tasks concurrently. I don't know how BOINC would react to that. Perhaps you are not doing that.

The hope would be that when Seti has work, the other two wouldn't have any. When Seti is out, the other two share the GPU resource and in so doing may end up doing a lot of work (ie not limited to 10%) because Seti outages seem to be frequent and protracted. The good thing would be that there shouldn't be excess tasks hanging around when Seti has work. Your 4 day cache would fill up immediately with Seti tasks and all would be good. Whilst I suspect that BOINC wouldn't handle the two backup projects very well, you should try it and see how close it gets to unattended operation.

On the point of cache size, I think your best chance of success would be to have a quite small overall size of say 0.25 days and be prepared to do a tiny bit of manual intervention. When Seti can't supply, BOINC will be prevented from over-fetching Einstein by the 0.25 day limit. As soon as Seti has work, some work will be fetched - enough to keep the machine crunching Seti (with its high resource share) until you notice. When you do notice, set NNT for the other projects and then set a large work cache size so that Seti can fill up to whatever the limit is. Then set the cache size back to 0.25 days and unset NNT. You should then have several days of unattended operation until the Seti cache is exhausted, at which time you could repeat the steps to refill with Seti if tasks were still available. If Seti announces a likely outage, you could do a 'fillup' before the outage commences. When all Seti tasks are finally exhausted and no more are available, the other projects will still be there to maintain 0.25 days of work.

Cheers,
Gary.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5875
Credit: 118476111518
RAC: 26010995

RE: ... I have set disk

Quote:
... I have set disk limits globally and locally and every possible permutation. I have tried local venues globally and locally. I have tried everything. If I have set a maximum disk usage for Einstein at 50MB, it downloads 2 GB of tasks ...


BOINC's disk limits mechanism sets a TOTAL for ALL projects. You can't set individual limits for EACH project. If you try to by setting certain limits at one particular project, all you will achieve is a new lower limit for ALL projects. When you go to change your computing preferences at a project website, read the very first line at the top which says, "These apply to all BOINC projects in which you participate". If you change in BOINC Manager rather than on a website, those changes will still propagate to all projects.

Quote:
... never finish in time because of the 1 week deadline and I have to abandon them.


The standard Einstein deadline is 2 weeks. If you have a 4 day setting and it's full with E@H tasks, why can't you finish them in 4 days - well within the deadline?

Quote:
If I have SETI getting 98% of resource time and Milkyway getting 1% and Einstein getting 1%, you would think that BOINC and the project would figure out the maximum allowed work onboard and restrict the number of tasks downloaded that can be completed within the deadlines. It does not.


If you have a 98/1/1 resource share, and if Seti can supply you would always have a cache full of Seti work with very little else. However, BOINC is designed to keep the cache full if it drops below the lo-water mark. If Seti can't supply, BOINC will take work from whatever project will supply and to hell with the consequences. You imply that the project should figure out that BOINC is asking for way too much and so should refuse supply. Unfortunately BOINC has not been designed to do this. By design, the project scheduler receives requests for work and attempts to honour them in full. The BOINC design does NOT require projects to restrict work supply. Some projects do restrict work supply but for entirely selfish reasons. Seti apparently does for the reason that it wants to share its limited availability as widely as possible and so prevent a limited number of crunchers from attempting to hog all the tasks. Milkyway does because (for reasons of efficiency in their modelling) they really would like the answers back immediately if not sooner. If you can only get a short time's worth of work, that's a pretty good method of preventing lots of old stale tasks being crunched too late to be really useful. BOINC is not able to properly handle projects that can't or wont supply on request. Why blame the project that is doing what the BOINC design expects? You should be taking out your frustration on the BOINC Devs for any perceived BOINC failings.

Cheers,
Gary.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5875
Credit: 118476111518
RAC: 26010995

RE: Well once again I have

Quote:
Well once again I have been foiled by BOINC. I once again tried to set venue preferences for Einstein and yet it seems to have affected SETI also in the default venue. I restricted E@H to 300MB of disk space for the Work venue.


Already explained. You CAN'T restrict just one project this way. It will restrict them all.


It might be a settings page, but it can't be yours. It will be the page for the person clicking the link, if they have a Seti account. Unfortunately I don't think the system will allow you to show your settings page to others.

Quote:
How can I restrict just Einstein and not have it affect my other projects?


You can't with disk quotas. You may be able to with the backup project mechanism or with manual adjustment of work cache settings at 'opportune' times.

Cheers,
Gary.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5875
Credit: 118476111518
RAC: 26010995

RE: OK, here is my request

Quote:
OK, here is my request to the developers. Set maximum amount of tasks a host can have onboard at any time just as SETI and MW do. They work correctly with BOINC.


OK, let's imagine for a moment that Einstein restricts the number of tasks on board to exactly the same number that Seti uses (whatever that is - I wouldn't have a clue). Now imagine that Seti is out of work and your 4 day cache is depleting fast. BOINC tries to get work from Seti but it can't. So it turns to Einstein and gets all it asks for. If the Seti limit allows you to fill a 4 day cache, wont the same limit at Einstein do exactly the same?? Oh, wait, you really didn't mean the same limit as Seti, you meant something like a 10 task limit, so it would seem. Yeah, that would be really reasonable for everybody else wouldn't it?

In summary, putting any sort of reasonable limit on Einstein cannot solve your particular issue - quite apart from the unfairness of requesting a limit in the first place.

Quote:
The deadlines for SETI are way more reasonable.


I would debate that - I would say they were more fit for purpose rather than reasonable. Seti's problem (apart from reliability) is that it has more requests for tasks than it is able to cope with. In the classic days it tried to cope (ie keep the punters happy) by reissuing the same work multiple times. I don't expect they are doing that these days. From the very little I know, they have tried to relieve pressure by encouraging willing participants to increasingly support other ventures.

If you think about it, having long deadlines will be helpful to the goal of having less participants continually 'banging on the doors for more work'. If you get a bunch of long deadline work, chances are that BOINC will tend to be forced into doing other projects' shorter deadline stuff first rather than the long deadline stuff that's not under pressure. That way BOINC will tend to do more of other projects and less of Seti, just to meet the deadlines. This would be particularly so for the typical case where Seti has by far the biggest resource share. So if you think that having long deadlines is great, it's not really if you mainly want to do Seti.

Quote:
Maybe set deadlines according to resource share for example. That would make much more sense in my opinion.


Not sure what you really mean by that?? Are you saying that if you (the volunteer) choose a 98/1/1 resource share, the task deadlines should be in the same ratio?? Or at least that the lower resource share projects should have a shorter deadline??

Deadlines are the same for all participants. If they get changed, everyone gets the same change irrespective of the project mix or resource share. There is no mechanism to change deadline per volunteer. There'd be a huge outcry if anything like that were attempted.

Cheers,
Gary.

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 5024
Credit: 18955486889
RAC: 6437495

RE: RE: Well once again I

Quote:
Quote:
Well once again I have been foiled by BOINC. I once again tried to set venue preferences for Einstein and yet it seems to have affected SETI also in the default venue.

On the account page for your Einstein (or your SETI) project account you have two links for preference settings. It is important for you to understand that one of the two is a generic BOINC preference which applies to ALL of your boinc projects, regardless of which account page you are looking at when you update it. The other, which is clearly labelled with the name of the project, contains project-specific settings.

Disk space is in the generic list, so is intended by the BOINC protocols to apply to all of your BOINC projects, not specifically the one from whose home page you revise it.

All fine and dandy but a small cache size just doesn't work for SETI. It would work quite well for MW and Einstein but you need a large cache of work for high powered computers that will crunch their daily quota in half a day because of SETI work unavailable every day because of their problems of not having enough work for all the crunchers.

Cheers, Keith

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 5024
Credit: 18955486889
RAC: 6437495

RE: I have no experience

Quote:

I have no experience with BOINC's backup project mechanism, but it seems to me that the only ways (without continual manual intervention) likely to stop BOINC filling up your 4 day cache with Einstein tasks is either, (a) don't have a 4 day cache, or (b) give Einstein a zero resource share. I'll deal with cache size later.

If you make Einstein a backup project, you probably wouldn't get any Einstein GPU tasks while Milkyway could supply them. So you would really need to make Milkyway a backup project too - if BOINC can have 2 backup projects (I don't know). If it can, and it's been set up correctly, 2 backup projects should share the resources "equally", depending on how BOINC defines "equally". Perhaps the GPU tasks would come alternately from either backup project which wouldn't be terribly "equal". A further complication would be if you were trying to run Einstein GPU tasks concurrently. I don't know how BOINC would react to that. Perhaps you are not doing that.

The hope would be that when Seti has work, the other two wouldn't have any. When Seti is out, the other two share the GPU resource and in so doing may end up doing a lot of work (ie not limited to 10%) because Seti outages seem to be frequent and protracted. The good thing would be that there shouldn't be excess tasks hanging around when Seti has work. Your 4 day cache would fill up immediately with Seti tasks and all would be good. Whilst I suspect that BOINC wouldn't handle the two backup projects very well, you should try it and see how close it gets to unattended operation.

On the point of cache size, I think your best chance of success would be to have a quite small overall size of say 0.25 days and be prepared to do a tiny bit of manual intervention. When Seti can't supply, BOINC will be prevented from over-fetching Einstein by the 0.25 day limit. As soon as Seti has work, some work will be fetched - enough to keep the machine crunching Seti (with its high resource share) until you notice. When you do notice, set NNT for the other projects and then set a large work cache size so that Seti can fill up to whatever the limit is. Then set the cache size back to 0.25 days and unset NNT. You should then have several days of unattended operation until the Seti cache is exhausted, at which time you could repeat the steps to refill with Seti if tasks were still available. If Seti announces a likely outage, you could do a 'fillup' before the outage commences. When all Seti tasks are finally exhausted and no more are available, the other projects will still be there to maintain 0.25 days of work.

Ok, I admit I've never noticed or paid attention to the text at the left. It just wasn't in my perception. I am paying attention to what I write. I apologize for hijacking the thread.

On the suggestion of setting a small cache size of 0.26 days, that just won't work for SETI. If I must do manual intervention to get work for SETI when it becomes available just to fill up my daily quota, what difference is it in just doing my manual intervention to get sufficient and not too much work for Einstein by setting NNT and the allowing a small amount of tasks to download at a time. Takes a lot less manual intervention since I only ask for work about every other day for Einstein when SETI has work. The only time I have to sit and monitor the computer is when SETI is out of work or down and my backup projects take over.

I will continue what I have been doing I guess and quit complaining about the inadequacies of BOINC.

Cheers, Keith

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 5024
Credit: 18955486889
RAC: 6437495

RE: RE: ... I have set

Quote:
Quote:
... I have set disk limits globally and locally and every possible permutation. I have tried local venues globally and locally. I have tried everything. If I have set a maximum disk usage for Einstein at 50MB, it downloads 2 GB of tasks ...

BOINC's disk limits mechanism sets a TOTAL for ALL projects. You can't set individual limits for EACH project. If you try to by setting certain limits at one particular project, all you will achieve is a new lower limit for ALL projects. When you go to change your computing preferences at a project website, read the very first line at the top which says, "These apply to all BOINC projects in which you participate". If you change in BOINC Manager rather than on a website, those changes will still propagate to all projects.

Quote:
... never finish in time because of the 1 week deadline and I have to abandon them.

The standard Einstein deadline is 2 weeks. If you have a 4 day setting and it's full with E@H tasks, why can't you finish them in 4 days - well within the deadline?

Quote:
If I have SETI getting 98% of resource time and Milkyway getting 1% and Einstein getting 1%, you would think that BOINC and the project would figure out the maximum allowed work onboard and restrict the number of tasks downloaded that can be completed within the deadlines. It does not.

If you have a 98/1/1 resource share, and if Seti can supply you would always have a cache full of Seti work with very little else. However, BOINC is designed to keep the cache full if it drops below the lo-water mark. If Seti can't supply, BOINC will take work from whatever project will supply and to hell with the consequences. You imply that the project should figure out that BOINC is asking for way too much and so should refuse supply. Unfortunately BOINC has not been designed to do this. By design, the project scheduler receives requests for work and attempts to honour them in full. The BOINC design does NOT require projects to restrict work supply. Some projects do restrict work supply but for entirely selfish reasons. Seti apparently does for the reason that it wants to share its limited availability as widely as possible and so prevent a limited number of crunchers from attempting to hog all the tasks. Milkyway does because (for reasons of efficiency in their modelling) they really would like the answers back immediately if not sooner. If you can only get a short time's worth of work, that's a pretty good method of preventing lots of old stale tasks being crunched too late to be really useful. BOINC is not able to properly handle projects that can't or wont supply on request. Why blame the project that is doing what the BOINC design expects? You should be taking out your frustration on the BOINC Devs for any perceived BOINC failings.

Then why the heck does the BOINC settings page offer venues which allow you to set individual parameters based on location. I set Einstein up for the Work venue with a much smaller disk space allotment. The page I was trying to show in the post for my account settings shows 4 columns of individual setting parameters. Default - HOME - SCHOOL - WORK. It shows the default location with my global 2 GB disk space limitation and for WORK venue it shows 300MB disk space allocation. On the details page for each computer working for Einstein I set the venue to WORK. When you first launch BOINC Manager, the log shows that the location for Einstein is WORK and it prints out the project parameters. Seems to me that is how the project is configured. Now everybody tells me that default or global overrides everything else, even the specific venue locations that you are able to input. Are you saying I have to set my other projects to one of the other venues and not assign them to default?? I don't think you can get rid of default, it is always present. I am sure I have tried this tactic before by assigning HOME to my other projects. It just does not work. I am most assuredly completely clueless on how BOINC and venues are supposed to work. What functionality are they supposed to provide?? They certainly don't provide the obvious interpretation in my case.

I ask again ... what purpose are the venues provided for?? Why do they even show separate disk space allocations if Default overrides them. How is anyone supposed to figure this out on their own??

Cheers, Keith

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 5024
Credit: 18955486889
RAC: 6437495

RE: RE: Well once again I

Quote:
Quote:
Well once again I have been foiled by BOINC. I once again tried to set venue preferences for Einstein and yet it seems to have affected SETI also in the default venue. I restricted E@H to 300MB of disk space for the Work venue.

Already explained. You CAN'T restrict just one project this way. It will restrict them all.


It might be a settings page, but it can't be yours. It will be the page for the person clicking the link, if they have a Seti account. Unfortunately I don't think the system will allow you to show your settings page to others.

Quote:
How can I restrict just Einstein and not have it affect my other projects?

You can't with disk quotas. You may be able to with the backup project mechanism or with manual adjustment of work cache settings at 'opportune' times.

Please show me the documentation that explains this so I can read it on my own and not ask obviously stupid questions in the forums??

Cheers, Keith

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 5024
Credit: 18955486889
RAC: 6437495

RE: RE: OK, here is my

Quote:
Quote:
OK, here is my request to the developers. Set maximum amount of tasks a host can have onboard at any time just as SETI and MW do. They work correctly with BOINC.

OK, let's imagine for a moment that Einstein restricts the number of tasks on board to exactly the same number that Seti uses (whatever that is - I wouldn't have a clue). Now imagine that Seti is out of work and your 4 day cache is depleting fast. BOINC tries to get work from Seti but it can't. So it turns to Einstein and gets all it asks for. If the Seti limit allows you to fill a 4 day cache, wont the same limit at Einstein do exactly the same?? Oh, wait, you really didn't mean the same limit as Seti, you meant something like a 10 task limit, so it would seem. Yeah, that would be really reasonable for everybody else wouldn't it?

In summary, putting any sort of reasonable limit on Einstein cannot solve your particular issue - quite apart from the unfairness of requesting a limit in the first place.

Quote:
The deadlines for SETI are way more reasonable.

I would debate that - I would say they were more fit for purpose rather than reasonable. Seti's problem (apart from reliability) is that it has more requests for tasks than it is able to cope with. In the classic days it tried to cope (ie keep the punters happy) by reissuing the same work multiple times. I don't expect they are doing that these days. From the very little I know, they have tried to relieve pressure by encouraging willing participants to increasingly support other ventures.

If you think about it, having long deadlines will be helpful to the goal of having less participants continually 'banging on the doors for more work'. If you get a bunch of long deadline work, chances are that BOINC will tend to be forced into doing other projects' shorter deadline stuff first rather than the long deadline stuff that's not under pressure. That way BOINC will tend to do more of other projects and less of Seti, just to meet the deadlines. This would be particularly so for the typical case where Seti has by far the biggest resource share. So if you think that having long deadlines is great, it's not really if you mainly want to do Seti.

Quote:
Maybe set deadlines according to resource share for example. That would make much more sense in my opinion.

Not sure what you really mean by that?? Are you saying that if you (the volunteer) choose a 98/1/1 resource share, the task deadlines should be in the same ratio?? Or at least that the lower resource share projects should have a shorter deadline??

Deadlines are the same for all participants. If they get changed, everyone gets the same change irrespective of the project mix or resource share. There is no mechanism to change deadline per volunteer. There'd be a huge outcry if anything like that were attempted.

OK, I admit you've shown how ludicrous my suggestions were.

Cheers, Keith

 

archae86
archae86
Joined: 6 Dec 05
Posts: 3161
Credit: 7282271708
RAC: 2039262

RE: I ask again ... what

Quote:
I ask again ... what purpose are the venues provided for?? Why do they even show separate disk space allocations if Default overrides them.


I think your error is in supposing that the venues are separately assignable by project.

Nope.

The model (BOINC's, not Einstein's) is that you assign a particular host to a particular location (aka venue) which then applies across all projects.

An extra detail is that one can over-ride location computing preferences for a single host by setting local preference on the host itself. This frees up the limitation of there being only four distinct locations, but it does not give you separate control by project, either.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.