[CLOSED] How to limit the amount of task I receive ?
7 Nov 2019 13:50:36 UTC
Topic 219935
(moderation:
Hello there,
I share my idle power with other project and I can limit each project's task maximum on them. I didn't find any options to do that here. Where can I find that option ?
To optimize each project's idle time, I limit them to 5 task each. So my computer can do a bit of all of them (I found that better than just letting BOINC switching task because every project get more work in general). But Einstein@home just send me again 75 task to complete. I know I can finish them each in ~7 minutes (for the GPU Gamma-ray pulsar binary search). But that lock my GPU for that project only. When I complete other project's task, my GPU is still used by einstein@home, so I don't get any other GPU task from other project. I think it's unfair to other project.
For now, I block new task from einstein@home and suspend a lot of them. Yesterday I even cancel some task. I just start using BOINC yesterday and I was lost with all that new task for these project alone.
Thanks
Hi Tutul, Welcome to the
)
Hi Tutul, Welcome to the Einstein project.
From the title of your thread, it seems like you want to limit each project based on a small number of tasks. BOINC does the 'limiting' and the normal method is to limit based on the expected time to complete rather than a specific number of tasks. You allocate the resource shares you wish each project to have. You also allocate an amount of time for which you want to keep a stock of tasks 'on hand'. BOINC then has the job of 'learning' how the apps for each project really perform so that it can download the correct number of tasks for each and allocate the correct sequence in which to process them to best meet your wishes. This could take quite a while (maybe even weeks) for BOINC to achieve (it depends on the number and complexity of projects you have) so you need to be patient. If you try to force BOINC to work 'differently' to the way that's it's designed to, you may very well end up making things worse - perhaps a lot worse.
One of BOINC's core features is the scheduling of work so that work can be completed efficiently and without risk of tasks missing deadlines. In the short term, it might seem like BOINC is doing the exact opposite of what you think it should do.
You have joined Einstein very recently. Your computers are 'hidden', which is the default setting. That's fine, but it means that if you want effective advice, you will need to spell out details about your computers, the projects you wish to run, which apps at each project you wish to run, what resource shares you wish to use, etc., etc. You have revealed one detail - you are running the gamma-ray pulsar search on GPUs and each task takes about 7 mins. That's great - you have a very nice GPU. If you want to limit that app to 5 tasks, you need to set your work cache size to 0.024 days! Imagine you had a different project/app where the tasks took 7 hours rather than 7 mins. To have 5 of those tasks, you would need a work cache size of 1.46 days. If your ultimate aim was for each project to share your resources equally, your best plan would be to set something like 0.5 to 1.0 days. At a 1 day setting, BOINC would try to download around 206 GRP tasks whilst for 7hr tasks it would allow 4 to be downloaded. However, BOINC would try to give each different app the correct amount of time to run so that it could meet your resource share specification.
This is just a trivial example - it can be a lot more complicated than that. Without knowing exactly the projects/apps you wish to run, the hardware details, the resource shares, etc., it's impossible to give proper advice.
There are much better options for achieving what you want. If you provide details, there are several people here who could give good advice on what would be best to do.
If you just abort tasks without reducing your work cache size, BOINC will just request more. You mentioned that at one point you had 75 GRP tasks. 75 x 7min = 8.75hrs. The deadline is 14 days. Why would you want to abort? Many of the hosts I run have hundreds of GRP tasks and no issues with returning them well within the deadline. Your best way to view things is on time to complete what you have and not the absolute number of them.
Cheers,
Gary.
Every other project I
)
Every other project I participate in give me a settings to limit the number of task I get. For example, the LHC@Home let me fix the maximum task I need to do for them at a fixed number or unlimited. I perfectly understand the "share" option BOINC use. I don't try to modify or BOINC behave. And the setting for work per day is a global one, not specific to a project.
I abort these task because Einstein@Home was locking my GPU for it only. In the event log, I read a lot of GPU task that was refuse for other project because my GPU was already used. If your project and BOINC download a full night of job for the GPU first, other project won't have a chance to use it.
Tutul wrote:Every other
)
Whilst some BOINC projects may have added features to do things like that, I don't think that most projects do. My understanding is that most projects don't have specific small limits like that. I could be wrong since it's a long time since I've run other projects.
There are fairly large daily limits to stop runaway hosts from trashing large numbers of tasks whilst still allowing a very fast host not to be overly restricted. By and large, it's up to the volunteer to set a work cache size that's suitable. Where is the need to set a 5 task limit if each task takes 7 mins? If the internet has a problem, or if the project has a temporary work shortage, particularly over a weekend, wouldn't it be better to have a somewhat bigger buffer?
Then why not give BOINC a chance to work out for itself how to fulfil your requirements? By aborting or suspending work that BOINC is trying to schedule, you are interfering with BOINC's ability to settle into a proper pattern. And, of course, the work cache setting is a global one. BOINC knows that it needs to keep a certain amount of work and it will schedule different projects to run so as to give each one its proper share of the resources - if you leave it alone for long enough to converge into a proper work allocation pattern without trying to force it.
E@H cannot lock your GPU. E@H has no control over when tasks are run. The ONLY thing E@H does is attempt to supply whatever work BOINC requests. BOINC is in control of which projects are asked to supply work and then in what order the tasks are run on your machine.
If you are certain that E@H is forcing extra tasks on your host so that other projects don't get asked for work, publish the event log to show this happening. You might find that BOINC asked another project first and that project couldn't supply. BOINC will then go through your list of projects in proper order until it finds one that can supply. Overall, this doesn't really matter. BOINC knows that E@H has had extra work done so will start actively trying to get work from whatever other projects have been 'missing out'. You just need to let BOINC sort it out rather than trying to force the issue. IF you really are sure that BOINC is giving Einstein too much use of your resources, reduce the E@H resource share and/or increase the shares of whatever GPU project you think is missing out. BOINC cannot instantly get everything running correctly but if you give it enough time and don't keep overriding its decisions, it will honour your requested shares in the longer term.
E@H doesn't force a "full night of job" on you. Please publish the event log that shows "refuse for other project because my GPU was already used." Also publish what came immediately before this "refuse" because that will probably show the correct reasons to account for what you see. You will find all the previous event log history in the file stdoutdae.txt in the BOINC data directory.
If you would like some help with really knowing what is happening and why, you need to publish all the details previously requested. At the moment people who would be able to make useful suggestions cannot do so. We have no idea of the details of your computer(s), the full range of projects you wish to support, or the resource shares you want each project to have.
Cheers,
Gary.
Other project are much slower
)
Other project are much slower (I've one that provide days long task) so the limit it's just their to prevent them to add too much task and let faster project to run (you're the only one where I go so fast).
I don't have the event anymore. My machine reboot since.
I checked some other project and you're right, most project doesn't seem to have that option. I was biased by my setup and I apologies for my lack of research.
I configured BOINC to give higher priority for the other project where my GPU was refuse. The project supply CPU and GPU for same task. Maybe the project perform an other check and, because the GPU was used, only provide CPU ?
After the new configuration, I'll see if anything appear in the event. For now, I just wait that the current pipeline finish to have a "like new" start.
Thanks you for you time. Maybe, as you said, it was just because BOINC was at is first launch.
Update: I want to apologies
)
Update: I want to apologies because, after checking every project I accepted, it's seems that the other one with GPU task don't have anymore GPU task since yesterday. Maybe I didn't understand BOINC message correctly.
Thanks for your time and, again, I apologies for my lack of research on the topic.
Btw, 70 GPU project for E@H it's about 8 hour of work, as I only run BOINC the night. The stat for every project show that E@H was almost the only one to complete work the first night. Now with better SHARE settings, every other project have completed task during the night too :)
Cheers
It would be interesting to
)
It would be interesting to see if Tutul has some new comments, now that some time has passed and he has passed and he's gained some BOINC experience.