All things Nvidia GPU

GWGeorge007
GWGeorge007
Joined: 8 Jan 18
Posts: 3104
Credit: 4988854142
RAC: 1316769

James Bradshaw wrote: Thanks

James Bradshaw wrote:

Thanks Mikey.  Understand, but I am running only two projects-Milky and Ein. I have tried to balance out all five hosts running both--CPU on all for Milky and GPU on all for Ein and I end up killing my production for both Mil and Ein.  Am going back to Ein only on four hosts and Milky on one host  That should keep my head above water for Ein.  I think in my case, I just need to save my pennies and get a host with a bunch of CPUs/cores dedicated for Milky...

Just a tip, since I'm not exactly sure what CPU cores/threads you are using... on each of your 5 computers.

Try this before you start spending money on something you won't need for a while.

For Milkyway CPU tasks you may want to run them at less than full CPU threads, maybe half... and still run Einstein GPU tasks on your GPUs.  This will become apparent to you as to why in a moment.

To get the most production out of your GPU's, run them at one task per GPU at a time to get a baseline of the time required to complete and validate the tasks for each GPU.  I would suggest getting 10 or more tasks completed and validated and then average them out for a baseline.

Once you have a baseline, then change them to two tasks per GPU and then check how long it takes and divide the time by 2.  (again, 10 or more)  If it is still less than your baseline, change them up to three tasks and check the times... and divide by three to compare with the baseline.  (same, 10 or more)

Continue on with this strategy until your multiple tasks per GPU give you a higher/longer timeline than your baseline.  If that's the case, revert to one less task per GPU to recover the best timeline per GPU on all GPUs.

After doing the above, you can now increase Milkyway tasks per CPU on each of the computers, one thread at a time, until your GPU timeline production starts to increase.  At that time reduce your CPU thread count by one to revert back to the GPU timeline per task you had before.

This could take a day or two to accomplish, but it should will give you the best times for both GPU and CPU on each of your computers.

Of course this does not include your using any one of the computers as a daily driver.  If you are, then I would suggest lessening the CPU usage on that particular computer by two threads.

If you are not sure how to change the number of tasks per CPU & GPU, or checking the timeline for tasks, just ask. 

We are here to help!

George

Proud member of the Old Farts Association

James Bradshaw
James Bradshaw
Joined: 1 Mar 14
Posts: 23
Credit: 1325617631
RAC: 2245606

George, thanks for your

George, thanks for your detailed thoughts and suggestions.  A number of months ago I did attempt to change the number of tasks for CPUs and GPUs, and the only thing that seemed to happen were messages that I was running multiple instances of BOINC..  What I did could be similar to a self-inflicted gun-shot wound...LOL.  So, some instruction for that would be most appreciated.  You already spent a lot of time on your last message, and I thank you for that.  JB

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4991
Credit: 18827216297
RAC: 5653899

Changing the number of WU's

Changing the number of WU's per gpu should not cause multiple clients to run.  You just need to change the divisor on the Project Preferences page for gpu utilization from 1.0 for one WU per gpu, 0.5 for two WU per gpu, 0.33 for three WU per gpu . . . etc etc

https://einsteinathome.org/account/prefs/project

At the bottom of this page is the configuration for each gpu application that Einstein offers.

 

KLiK
KLiK
Joined: 1 Apr 14
Posts: 67
Credit: 447179111
RAC: 1012236

James Bradshaw wrote: Mikey,

James Bradshaw wrote:

Mikey, more specific to your comment, yes, I can leave the MX550 host on EIN only, so that is a good suggestion.  I have had a problem balancing out usage of my hosts with Milky going CPU only and Ein doing GPU.  Perhaps my problem involves that Ein GPU tasks also use at least some CPU time??  My overall performance took a big hit with Milky/CPU and Ein/GPU conflicts.  

Note that running BRP7 usually takes quite less CPU power (0,2 CPU power in my example) , then ASGWS-O3.

& also, I run my ASGWS-O3 with only 0,9 on CPU, in order to keep other CPU tasks running (on WCG in my example).

Give it a try, let us know how it works!

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6504
Credit: 9616043476
RAC: 3426564

James Bradshaw wrote:Mikey,

James Bradshaw wrote:

Mikey, more specific to your comment, yes, I can leave the MX550 host on EIN only, so that is a good suggestion.  I have had a problem balancing out usage of my hosts with Milky going CPU only and Ein doing GPU.  Perhaps my problem involves that Ein GPU tasks also use at least some CPU time??  My overall performance took a big hit with Milky/CPU and Ein/GPU conflicts.  

Every GPU task uses some CPU resources.

I know that previous experience with running Intel iGpu's slows CPU processing speeds running at the same time.

I suspect this is a limitation caused by running the iGpu on CPU memory and other bandwidth limits.

AMD iGpu's may have the same architectural limitations.

Tom M

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6504
Credit: 9616043476
RAC: 3426564

I have been poking around

I have been poking around trying to find an explanation for how the parameter: ACTIVE THREAD effects things.

 

CUDA_MPS_ACTIVE_THREAD_PERCENTAGE

On Volta GPUs, this environment variable sets the portion of the available threads that can be used by the client contexts. The limit can be configured at different levels

.So what does this actually mean?

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 4008
Credit: 47587725895
RAC: 44823566

In simple terms it allocates

In simple terms it allocates the percentage of available threads or SMs from the GPU. Say you have 5000 threads (cores) on the GPU. And you set the active thread percentage to 50%, each process will only use up to 2500 threads. 

_________________________________________________________________________

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6504
Credit: 9616043476
RAC: 3426564

Ian&Steve C. wrote: In

Ian&Steve C. wrote:

In simple terms it allocates the percentage of available threads or SMs from the GPU. Say you have 5000 threads (cores) on the GPU. And you set the active thread percentage to 50%, each process will only use up to 2500 threads. 

So 70% would provide (potentially) higher/faster processing than 40%?  And 100% would basically put you back to straight round robin time slicing?

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 4008
Credit: 47587725895
RAC: 44823566

You have to test to find

You have to test to find out. 
 

if you were only running 1x task then yes 70% would probably be faster. But with running 4-5x I think 40% is faster 

_________________________________________________________________________

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6504
Credit: 9616043476
RAC: 3426564

A solid case can be made that

A solid case can be made that a Titan V is more "efficient" (uses less electricity for a given level of production) than a EVGA rtx 3080 ti.  But what happens to the rtx 3080 ti's production if you were to power limit it to say 200 watts?

My Titan V's are running ~150 watts or less.  If I can get the equivalent production at 200 watts on my rtx 3080 ti running the same number of threads....

Experiment has started.

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.