Daily task quota

Tom M
Tom M
Joined: 2 Feb 06
Posts: 2,204
Credit: 3,722,759,274
RAC: 5,550,996

DanNeely wrote: Going the

DanNeely wrote:

Going the fake GPU route I've actually had the opposite problem.  There's a separate clientside limit on max tasks at a time, and my local backlog has become entirely filled with GPU tasks, leaving my CPUs running a backup project for the last few days.

I think I'm going to try switching to fake CPUs instead and see if skewing the notionally desired workload mix will help.  If it doesn't help, or if it ends up skewing the problem hard the other direction, I guess I'll have to reduce my desired amount of local data.  I don't really like doing so; but unless someone can point me to another setting to fiddle it doesn't look like I can get enough GPU work to have a few days on hand no matter what I do.

 

10/31/2021 6:40:50 PM | Einstein@Home | update requested by user
10/31/2021 6:40:50 PM | Einstein@Home | [sched_op] sched RPC pending: Requested by user
10/31/2021 6:40:50 PM | Einstein@Home | [sched_op] Starting scheduler request
10/31/2021 6:40:51 PM | Einstein@Home | Sending scheduler request: Requested by user.
10/31/2021 6:40:51 PM | Einstein@Home | Reporting 6 completed tasks
10/31/2021 6:40:51 PM | Einstein@Home | Not requesting tasks: too many runnable tasks
10/31/2021 6:40:51 PM | Einstein@Home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
10/31/2021 6:40:51 PM | Einstein@Home | [sched_op] NVIDIA GPU work request: 0.00 seconds; 0.00 devices
10/31/2021 6:40:54 PM | Einstein@Home | Scheduler request completed
 

When I end up with a message like "too many runnable tasks" the simplest way to "clear" things seems to be "reset" the project.  I would try to make the changes that will keep you from getting "to many runnable tasks" first though.

I have been running days of work and additional days of work at 0.1 which might help.

Using the <ncpu>16</ncpu> (on an 4c/8 cpu) in the cc_config.xml file seems to be the most straight forward way to lift the daily task limit for me. I am not having that issue right now.  But when I was playing with high multiples of gpus cards it certainly bit me.

Tom M

A proud member of the O.F.A. (Old Farts Association)

My other CPU is a Ryzen 3950x

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 2,314
Credit: 5,941,428,534
RAC: 14,100,292

Why are you concerned about

Why are you concerned about bunkering several days of work at Einstein?

They have very stable servers and have never had any outages that I can remember when they couldn't deliver work when asked.

I just operate on a report one/ask for one replacement schedule and my hosts always have work.

I keep just 30 tasks on hand at all times.  No issues whatsoever.

 

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1,359
Credit: 2,921,351,593
RAC: 2,990,869

Keith Myers wrote: Why are

Keith Myers wrote:

Why are you concerned about bunkering several days of work at Einstein?

They have very stable servers and have never had any outages that I can remember when they couldn't deliver work when asked.

I just operate on a report one/ask for one replacement schedule and my hosts always have work.

I keep just 30 tasks on hand at all times.  No issues whatsoever.

 

My ISP can be slow to repair things if it doesn't affect a lot of customers.  ex earlier this year someone (99% sure a neighbor) bumped a utility pole right were the connection between the coax coming down the pole and the coax going into my building were; breaking it badly enough I couldn't kludge up a temporary repair.  I called them first thing Monday morning.  It was Wednesday afternoon before they were able to send someone out to fix it.

 

Until my GPUs got fast enough to run into work unit limits I liked to keep a few days of work on hand to cover both stuff like that, and cases when the WU generators run dry just before a weekend.  I still want to be able to cover shorter maintenance/repair outages even if I can't do that anymore.

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 2,314
Credit: 5,941,428,534
RAC: 14,100,292

Solution for temp outages at

Solution for temp outages at Einstein is to have some backup projects.

 

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 1,749
Credit: 13,285,156,014
RAC: 40,377,128

As this is the most recent

As this is the most recent thread about this issue and with a relevant title, I figured I would post an update. I contacted Bernd and asked him to review and update host daily quota limits, and he has essentially doubled the limits. 
 

Quote:

I doubled the "daily_result_quota" (per CPU) from 32 to 64, which should double the GPU task limit automatically, and "max_ncpus" as well (may need some minutes to propagate). AFAIK there is no adjustable limit for the number of GPUs of a host; if there is one, it's probably hardcoded (table size) in the client an the server as well and can't easily be changed.

 

BM


 

so new limits should be 64 tasks per CPU, up to 128 CPUs. and 512 tasks per GPU, up to 8 GPUs. 
 

most people will not have to adjust any settings anymore unless someone is running a really fast GPU with low core count CPU. 

_________________________________________________________________________

archae86
archae86
Joined: 6 Dec 05
Posts: 3,066
Credit: 5,901,839,242
RAC: 3,237,219

Ian&Steve C. wrote:so new

Ian&Steve C. wrote:

so new limits should be 64 tasks per CPU, up to 128 CPUs. and 512 tasks per GPU, up to 8 GPUs. 
 

most people will not have to adjust any settings anymore unless someone is running a really fast GPU with low core count CPU. 

Good news.  I have so far changed my falsified ncpus value in cc_config.xml on one machine from 24 to 0.

By the way, several people posting here have cited "-1" as the proper ncpus value to get the real detected number into use, but the BOINC online documentation at:

https://boinc.berkeley.edu/wiki/Client_configuration

specifies "0" for this purpose, so I tried that first.  As it should have, after doing Options|read config files, followed by an update in BOINC, that got me the expected value of 4 displayed on my account web page.

Perhaps both "-1" and "0" work?  Or perhaps "-1" was a mistaken advice?

 

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 1,749
Credit: 13,285,156,014
RAC: 40,377,128

The client probably just

The client probably just ignores the -1 value and reads the number from the system. Personally I just remove that element from the cc_config file altogether which achieves the same end. 

_________________________________________________________________________

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 2,314
Credit: 5,941,428,534
RAC: 14,100,292

The default value is -1 from

The default value is -1 from a virgin cc_config initialization.  The value is pulled from the client source code parameter setting.

 

tictoc
tictoc
Joined: 1 Jan 13
Posts: 16
Credit: 4,143,628,536
RAC: 1,490,033

Ian&Steve C. wrote:As this

Ian&Steve C. wrote:

As this is the most recent thread about this issue and with a relevant title, I figured I would post an update. I contacted Bernd and asked him to review and update host daily quota limits, and he has essentially doubled the limits. 
 

Quote:

I doubled the "daily_result_quota" (per CPU) from 32 to 64, which should double the GPU task limit automatically, and "max_ncpus" as well (may need some minutes to propagate). AFAIK there is no adjustable limit for the number of GPUs of a host; if there is one, it's probably hardcoded (table size) in the client an the server as well and can't easily be changed.

 

BM


 

so new limits should be 64 tasks per CPU, up to 128 CPUs. and 512 tasks per GPU, up to 8 GPUs. 
 

most people will not have to adjust any settings anymore unless someone is running a really fast GPU with low core count CPU. 

 

Thanks for taking the initiative on this, and thanks to the admin for making the change.

 

On a side note, sorry to any of my wingmen who were caught waiting for validation due to two driver crashes that resulted in a few thousand of my tasks erroring out last week.   

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.