My GPU isn't saturated by the arecibo period search application. Nvidia-smi says <45% is used, though it varies. What can I do to make sure it is saturated?
My GPU isn't saturated by the arecibo period search application. Nvidia-smi says <45% is used, though it varies. What can I do to make sure it is saturated?
webpage, your account, preferences for Einstein:
GPU utilization factor of BRP apps
DANGEROUS! Only touch this if you are absolutely sure of what you are doing!
Wrong setting might even damage your computer! Use solely on your own risk!
Min: -1.0 / Max: 1.0 / Default: 1.0, negative values will disable GPU tasks of this type 1.0
Change the 1.0 to 0.5 and you will begin running 2 units at once, change it to 0.33 and you will run 3 units at a time. The more you run the more your gpu will be used, BUT there is a point, each machine and gpu is different, where running too many units at once slows everything and it's overloaded. Start with a setting of 0.5, you MUST leave a cpu core free just to keep the gpu fed to make all this work, and see what your gpu load is and your times. Run it that way for say 10 units or a day, whatever works for you, then switch it too 0.33 and do the same thing, again writing down the times for each unit. At some point it will get slower nad if it's your only pc get too laggy, at that point back off and let it run and see if you are happy. Remember though running more than one unit at a time WILL increase the heat output of your gpu, so make sure you have adequate cooling.
You are NOT looking for a 100% load, because then it will run out of memory when it switches to the next unit, but the high 80's or low 90's gpu load is okay. Again though watch the unit times and gpu temp, as long as you are ahead by running 2 at a time over running just one at a time it's all good. Same thing with 3 units at once over 2 units at once, but once the unit times go negative, ie it is taking longer to run than the savings you are getting by running multiple units, then back off. IE--let's say 1 unit takes you 90 minutes to finish, these are just numbers, when running 1 unit at a time and running 2 units at a time you finish both of them in only 150 minutes. Then you are ahead as 2 units running 1 at a time would take 180 minutes, 90 minutes each.
I like MsiAfterburner for my gpu temp monitoring app, it works on both AMD and Nvidia cards and it will automatically speed up the fan if the gpu starts getting too hot. You can either let the default fan settings handle it or use your own settings, I use the defaults.
Initially, I had three Units running at a time; but, there's been a change in work units. One type has been phased out, and another, (new), Unit brought in. The new Units are taking longer to crunch on my GTX-760; so, I backed off from .33 to .5, and now only crunch two Units at a time.
My GPU isn't saturated by the arecibo period search application. Nvidia-smi says <45% is used, though it varies. What can I do to make sure it is saturated?
... you MUST leave a cpu core free just to keep the gpu fed to make all this work ...
The truth of the matter is that, for NVIDIA GPUs, in most cases reserving a CPU core doesn't make much difference (apart from reducing your CPU crunching output), particularly for running only 2 concurrent tasks on the GPU. This was true for the previous BRP5 GPU app and will apply even more so for the new BRP6-beta app which will become the standard app once the beta test is completed. You could get an immediate improvement with your current setup if you changed your preferences to allow beta test work. The BRP6 tasks list on one of your computers shows quite long run times for BRP6 tasks using the old app, with quite a heavy CPU component. I would expect both figures to show a nice reduction with the beta-1.52 app.
BOINC sees the GPU as "NVIDIA GRID K520 (133484543MB)". The RAM size is obviously wrong and you might get a better detection of the GPU with a more recent BOINC than 7.0.27. I know nothing about that GPU but the run times indicate it is not very powerful for crunching purposes. It is definitely worthwhile changing the GPU utilization factor as Mikey suggested to allow at least 2 concurrent tasks. Once you have found the optimum task concurrency, you should do the experiment to see if freeing a CPU core makes any further improvement. You wont know until you try.
BOINC sees the GPU as "NVIDIA GRID K520 (133484543MB)". The RAM size is obviously wrong and you might get a better detection of the GPU with a more recent BOINC than 7.0.27. I know nothing about that GPU but the run times indicate it is not very powerful for crunching purposes.
Looking at the list of computers (identical) and knowing that the GRID series are capable of being virtualized I think Tim is running several (3 by the looks of it) virtual machines against one GPU.
This could explain why the GPU is not performing as you'd expect with a native GPU.
... I think Tim is running several (3 by the looks of it) virtual machines against one GPU.
This could explain why the GPU is not performing as you'd expect with a native GPU.
Fair enough. I know nothing about virtual machines - I've not even considered running one. If he was running 3 virtual machines against 1 GPU would that mean he was already doing the equivalent of running 3 concurrent GPU tasks in a 'real' machine? If he must run 3 virtual machines (for whatever other reasons), would he be better off performance wise just crunching on one and running concurrent GPU tasks on that one?
My only experience of virtual machines and GPUs is dedicating a GPU to a single virtual machine, essentially running the GPU native. I have not run an advanced setup like GRID and a virtualized GPU.
Quote:
If he was running 3 virtual machines against 1 GPU would that mean he was already doing the equivalent of running 3 concurrent GPU tasks in a 'real' machine?
I would think so, but with all the overhead the virtualization layer would give.
Quote:
If he must run 3 virtual machines (for whatever other reasons), would he be better off performance wise just crunching on one and running concurrent GPU tasks on that one?
I guess the answer is in how good the Nvidia driver is at virtualizing the GPU.
Since virtualizing GPUs is a new technology I assume the drivers are immature and probably optimized for graphics performance, not computing.
GPU not saturated
)
webpage, your account, preferences for Einstein:
GPU utilization factor of BRP apps
DANGEROUS! Only touch this if you are absolutely sure of what you are doing!
Wrong setting might even damage your computer! Use solely on your own risk!
Min: -1.0 / Max: 1.0 / Default: 1.0, negative values will disable GPU tasks of this type 1.0
Change the 1.0 to 0.5 and you will begin running 2 units at once, change it to 0.33 and you will run 3 units at a time. The more you run the more your gpu will be used, BUT there is a point, each machine and gpu is different, where running too many units at once slows everything and it's overloaded. Start with a setting of 0.5, you MUST leave a cpu core free just to keep the gpu fed to make all this work, and see what your gpu load is and your times. Run it that way for say 10 units or a day, whatever works for you, then switch it too 0.33 and do the same thing, again writing down the times for each unit. At some point it will get slower nad if it's your only pc get too laggy, at that point back off and let it run and see if you are happy. Remember though running more than one unit at a time WILL increase the heat output of your gpu, so make sure you have adequate cooling.
You are NOT looking for a 100% load, because then it will run out of memory when it switches to the next unit, but the high 80's or low 90's gpu load is okay. Again though watch the unit times and gpu temp, as long as you are ahead by running 2 at a time over running just one at a time it's all good. Same thing with 3 units at once over 2 units at once, but once the unit times go negative, ie it is taking longer to run than the savings you are getting by running multiple units, then back off. IE--let's say 1 unit takes you 90 minutes to finish, these are just numbers, when running 1 unit at a time and running 2 units at a time you finish both of them in only 150 minutes. Then you are ahead as 2 units running 1 at a time would take 180 minutes, 90 minutes each.
I like MsiAfterburner for my gpu temp monitoring app, it works on both AMD and Nvidia cards and it will automatically speed up the fan if the gpu starts getting too hot. You can either let the default fan settings handle it or use your own settings, I use the defaults.
Initially, I had three Units
)
Initially, I had three Units running at a time; but, there's been a change in work units. One type has been phased out, and another, (new), Unit brought in. The new Units are taking longer to crunch on my GTX-760; so, I backed off from .33 to .5, and now only crunch two Units at a time.
TL
TimeLord04
Have TARDIS, will travel...
Come along K-9!
Join SETI Refugees
RE: RE: My GPU isn't
)
The truth of the matter is that, for NVIDIA GPUs, in most cases reserving a CPU core doesn't make much difference (apart from reducing your CPU crunching output), particularly for running only 2 concurrent tasks on the GPU. This was true for the previous BRP5 GPU app and will apply even more so for the new BRP6-beta app which will become the standard app once the beta test is completed. You could get an immediate improvement with your current setup if you changed your preferences to allow beta test work. The BRP6 tasks list on one of your computers shows quite long run times for BRP6 tasks using the old app, with quite a heavy CPU component. I would expect both figures to show a nice reduction with the beta-1.52 app.
BOINC sees the GPU as "NVIDIA GRID K520 (133484543MB)". The RAM size is obviously wrong and you might get a better detection of the GPU with a more recent BOINC than 7.0.27. I know nothing about that GPU but the run times indicate it is not very powerful for crunching purposes. It is definitely worthwhile changing the GPU utilization factor as Mikey suggested to allow at least 2 concurrent tasks. Once you have found the optimum task concurrency, you should do the experiment to see if freeing a CPU core makes any further improvement. You wont know until you try.
Cheers,
Gary.
RE: BOINC sees the GPU as
)
Looking at the list of computers (identical) and knowing that the GRID series are capable of being virtualized I think Tim is running several (3 by the looks of it) virtual machines against one GPU.
This could explain why the GPU is not performing as you'd expect with a native GPU.
RE: ... I think Tim is
)
Fair enough. I know nothing about virtual machines - I've not even considered running one. If he was running 3 virtual machines against 1 GPU would that mean he was already doing the equivalent of running 3 concurrent GPU tasks in a 'real' machine? If he must run 3 virtual machines (for whatever other reasons), would he be better off performance wise just crunching on one and running concurrent GPU tasks on that one?
Cheers,
Gary.
My only experience of virtual
)
My only experience of virtual machines and GPUs is dedicating a GPU to a single virtual machine, essentially running the GPU native. I have not run an advanced setup like GRID and a virtualized GPU.
I would think so, but with all the overhead the virtualization layer would give.
I guess the answer is in how good the Nvidia driver is at virtualizing the GPU.
Since virtualizing GPUs is a new technology I assume the drivers are immature and probably optimized for graphics performance, not computing.
Thanks for responding. Maybe
)
Thanks for responding. Maybe the OP can tell us all about it :-).
Cheers,
Gary.