Cuda and priorities
24 Feb 2012 18:28:30 UTC
Topic 196199
(moderation:
I'm running the Binary Radio Pulsar Search (Arecibo) cuda app on Scientific Linux 6.2 (Kernel 2.6.32-220.4.1.el6.x86_64 #1 SMP) with Boinc 6.10.45.
Suspending when computer is in use has been broken for Linux in Boinc since... well... forever. (Unreleased 6.13.something includes the simple fix. But I prefer running only distro repository packages.)
So I just use a simple script calling xprintidle and boinccmd to do this externally for my cuda app. But my CPU apps running at SCHED_IDLE are no problem at all to keep running all the time. I'd like to run the cuda app in a similar fashion, if practical.
However, the cuda app still causes performance problems for Flash videos, VPython animations, etc. Not surprising since SCHED_IDLE only controls CPU time and not GPU time. Is there some way to set a GPU priority or cuda priority to make the cuda app 'nicer'?
Thanks,
Steve Bergman
Cuda and priorities
)
You could try and set the compute mode for the card to exclusive process which makes it so only one process can use the GPU at a time.
nvidia-smi -L
nvidia-smi -i 0 -c 3
The first command gives you a list of GPUs installed and the -i flag is used to target a GPU.
The CUDA app is dependent on the CPU to an extend so renicing the process might make a difference also. Something like this via a cronjob:
renice PRIO -p `ps --no-headers -Lo lwp -p $(pgrep -f BRP3cuda)`
Another thing you might look at doing is running nvidia-smi -q and then grabbing the GPU usage. In general the Einstein CUDA app does not put the GPU under 100% load. Based on a higher load due to running other GPU intensive programs, you could send a SIGSTOP (kill -19) to the BRP CUDA process to temporarily pause the process. Then when the load is back down, send a SIGCONT (kill -18) to the BRP CUDA process.
Hi! I don't think sending
)
Hi!
I don't think sending SIGSTOP to the apps is a good idea. When the app comes back to life, it will probably notice that it hasn't received a "heart beat" message from the core client for some time and will exit! Suspending via boinccmd as mentioned above is a better way IMHO. Setting the card to exclusive mode will (if this works at all) probably also not improve responsiveness as well in the absence of a priority scheme (how should OS know it should favor the (say) video playing task and not the BOINC one when both compete for the GPU?).
If you are into scripting, you might want to try to use an existing X screensaver for idle detection (needless to say you should use one that doesn't put load on the GPU :-) ) and to drive your scripts. See the "-watch" option here:
http://www.jwz.org/xscreensaver/man3.html
CU
HBE
Thank you for the responses.
)
Thank you for the responses. I suspected that GPU process scheduling wasn't in a very advanced state. But I figured I'd ask.
I'm running the process in SCHED_IDLE class (chrt -i) but that doesn't seem to help appreciably.
The small but somewhat hard to find utility, xprintidle.c, uses the the libX11 routines for determining idle time. You just run it, and it reports the number of milliseconds since a key has been hit or the mouse has been moved. It's what I'm using in my script. So I guess my "solution" is the best for now.
-Steve
I just suspend the GPU work
)
I just suspend the GPU work through the client if I am doing anything demanding. It seems to be easy enough to do manually.