Running multiple tasks concurrently per GPU - How to revert to just one task.

mikey
mikey
Joined: 22 Jan 05
Posts: 12509
Credit: 1838553268
RAC: 3680

KLiK wrote: Keith Myers

KLiK wrote:

Keith Myers wrote:

You need to try to stagger the task startups on the gpu so that they all don't converge at the 49% and 99% cpu offload points when running gpu task multiples.

Just suspend a task that starts anew on the gpu until the previous one had made progress halfway to one of its cpu compute intervals, then un-suspend the task.

Thank you.

Immediate result can be seen:

 

It sure looks like both tasks have started and running in sync instead of opposite to each other

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3896
Credit: 42978642644
RAC: 60486022

mikey wrote: It sure looks

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles.

_________________________________________________________________________

mikey
mikey
Joined: 22 Jan 05
Posts: 12509
Credit: 1838553268
RAC: 3680

Ian&Steve C. wrote: mikey

Ian&Steve C. wrote:

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles. 

okay thanks

KLiK
KLiK
Joined: 1 Apr 14
Posts: 56
Credit: 345727324
RAC: 1319964

mikey wrote:Ian&Steve C.

mikey wrote:

Ian&Steve C. wrote:

mikey wrote:

It sure looks like both tasks have started and running in sync instead of opposite to each other



you're focusing on the wrong part of the graph. the beginning is with the tasks not staggered, but the end shows the second task start right when the first task ends, leading to near constant GPU utilization. you can see a slight dip in temp, then it comes right back up instead of falling down like the previous two cycles. 

okay thanks

Its like @mikey said. 

But also, there are 2 cards in the system. So both cards work, that is why there are 2 graphs!

(Though my M5000 & P4 was read with only single temp., so I changed from M5000 to P2000...waiting for Eintein@home tasks to start again, to show the real results.)

Main thing is: previous dips were after around 18~20min of GPU & lasted for about ~15min with CPU time. If I run 2x per GPU, then the 2nd task can almost be over by the time 1st one finished it CPU calc. ????

KLiK
KLiK
Joined: 1 Apr 14
Posts: 56
Credit: 345727324
RAC: 1319964

Keith Myers wrote: Yes, same

Keith Myers wrote:

Yes, same principle. But in my experience so far with MeerKAT tasks, they don't respond as well to 2X as the optimized app for GR#1.  You won't see as much benefit . . . .  if any.

They also use  more VRAM.  2X may be cutting it close with 6GB.

Are you sure about that? This is MeerKAT, with only 22~25% load & not much memory used.

Did anybody use 2x WU on stronger cards, like Teslas - which can crunch more data? What are the experiences? 

GWGeorge007
GWGeorge007
Joined: 8 Jan 18
Posts: 2994
Credit: 4925574443
RAC: 355702

KLiK wrote: Keith Myers

KLiK wrote:

Keith Myers wrote:

Yes, same principle. But in my experience so far with MeerKAT tasks, they don't respond as well to 2X as the optimized app for GR#1.  You won't see as much benefit . . . .  if any.

They also use  more VRAM.  2X may be cutting it close with 6GB.

Are you sure about that? This is MeerKAT, with only 22~25% load & not much memory used.

Did anybody use 2x WU on stronger cards, like Teslas - which can crunch more data? What are the experiences? 

Your image doesn't show much of a load from a Tesla card, and your two Tesla K20Xm's have less than 6GB of VRAM in Windows 10 running an Intel i7-5820K CPU.  It may matter what your PC & BOINC specs are set to.

George

Proud member of the Old Farts Association

Marcelo
Marcelo
Joined: 14 Dec 11
Posts: 7
Credit: 94062114
RAC: 1594630

Hello friends of

Hello friends of science,

 

so far I have only been working with “simple” PC builds and found information about the settings relatively quickly in the forum. Now I wanted to integrate a second graphics card and run both in parallel. 

 

I activated multiGPU via cc_config as follows. 

 

 

  <cc_config>                                       
             <options>                                   
                                                                                                           
                      <use_all_gpus>1</use_all_gpus>           
                                                       
             </options>                                 
  </cc_config>

 

 

 

Because my 1st GPU (RTX 4080) has enough VRAM, I have set several WU per GPU via app_config.

 

 

<app_config>
    <app>
        <name>einstein_O3AS</name>
        <gpu_versions>
            <gpu_usage>0.25</gpu_usage>
            <cpu_usage>1</cpu_usage>
        </gpu_versions>
    </app>
</app_config>

 

 

 

My question:

 

I want the 1st GPU (RTX 4080) to calculate 4 WUs at the same time and the 2nd GPU (RTX 2070 Super) to calculate 3 WUs. Because the 2070 has too less VRAM and can only load 3 WUs.

 

How do I have to set this up?

 

Regards

 

Marcelo

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3896
Credit: 42978642644
RAC: 60486022

Marcelo wrote: My

Marcelo wrote:

My question:

I want the 1st GPU (RTX 4080) to calculate 4 WUs at the same time and the 2nd GPU (RTX 2070 Super) to calculate 3 WUs. Because the 2070 has too less VRAM and can only load 3 WUs.

How do I have to set this up?

Regards

Marcelo


the short answer is you can't*

*the longer answer is you can, but only if you run two instances of BOINC with each GPU dedicated to a separate instance, or put the GPUs in separate computers

_________________________________________________________________________

Marcelo
Marcelo
Joined: 14 Dec 11
Posts: 7
Credit: 94062114
RAC: 1594630

Okay thanks for the quick

Okay thanks for the quick info. 

And is it possible that one GPU only processes O3ASHF1d WUs and the other GPU only executes Binary Radio Pulsar Search (MeerKAT) WUs?


I'm currently testing which configuration has the best efficiency (power consumption/crunch power/WUs etc :D)

 

 

 

 

San-Fernando-Valley
San-Fernando-Valley
Joined: 16 Mar 16
Posts: 337
Credit: 8979443455
RAC: 10220442

Check the exclude option

Check the exclude option (etc.) described in the User Manual under cc_config.

Might be of interest.

SFV

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.