GPU Utilization question
29 Apr 2023 22:21:49 UTC
Topic 229457
(moderation:
Hello, I am new to distributed computing, but I think I'm hooked. I don't know that much about computers, but this has inspired me to learn more and even dip my toe into overclocking.
I have a question about how the GPU Utilization setting is handled. In particular, if you enter a number that has a remainder, how is that handled? TLDR: if I enter 0.3, will that make me always have 3 jobs or will it sometimes try to load 4?
More information: When doing one GPU job at a time I get ~4GB/11GB of VRAM used and ~40% GPU utilization in MSI Afterburner. I took the plunge and have set it to 0.5, which makes it so I have ~7 GB and ~80% utilization when both tasks are GPU crunching at the same time (on my hardware it takes ~45 seconds to start a job while it is loaded into VRAM, then ~5 minutes of GPU crunching to get it to 99%, then ~4 minutes of wrapup where the job is still in memory but is not utilizing the GPU, for a total of 10-10.5 minutes per GPU job). I figure I can get away with 3 jobs at once if I stagger them so only 2 are GPU crunching at once. My hardware makes it pretty natural that 50% of time is utilizing GPU and 50% isn't, so as long as I set the timing right initially I should be golden. I'd love to go for four, but I don't think I can fit that in VRAM.
It is is hard to maintain a
)
It is is hard to maintain a stagger in task startup due the variability of task compute length. Invariably, you will encounter multiple task launch occurring at the same time.
Gpu utilization at 0.5X is safest. You might get away with 0.33X though.
No remainder occurs. It is only an integer value of number of tasks.
Thank you for the quick
)
Thank you for the quick reply. Three at a time may be pushing it. You are right that eventually they will get out of sync with each other. I was planning on a new GPU once the 4090Ti inevitably comes out, I may have to go for two.
Jimmothy wrote: Thank you
)
The 40?? series of gpu's currently has a very high error rate here at Einstein but the Admins are working on it so you might want to slow your roll on those purchases for the time being and let the others with them do the testing. Once they are done you can then swoop in and take over the stats sheets!!
As an aside be sure you keep enough cpu cores free from cpu crunching to keep the gpu you do have fed and running at full speed.
That is a fair point. I'm
)
That is a fair point. I'm guessing the 40xxTi won't come out until the end of 2023, so it should be plenty of time to work the bugs out.
I have just been using the settings on the webpage. It looks like I can have some more fine control by editing .xml files. I did set swan_sync 0, but there is so much speculation about that technique that I have a sneaking suspicion it may be bro science.
Jimmothy wrote: That is a
)
You have your pc hidden so I have to ask you what OS are you running? If Linux there is an optimized app to run Einstein tasks but their is not a Windows one. Search for the Lunatics app
Darn shame about that. I
)
Darn shame about that. I generally believe in more privacy than not, but my computer should be visible now. Also, what are the odds of me happening to refresh this page 5 minutes after you replied, eh?
EDIT: I happen to have a Ryzen R9 CPU, which seems to be very good at doing a lot of stuff at once, but not particularly good at doing anything fast. 25 simultaneous tasks that take 12 hours to complete.
EDIT AGAIN:
That image in your signature is quite impressive. I recognize the leftmost one from GPUGRID, but may I have a breakdown of the other racks you have there?
Jimmothy wrote: Darn shame
)
Sure https://stats.free-dc.org/stats.php?page=userbycpid&cpid=da0c795e81490ce47752d88cb0cbd5be
I have been crunching since the early days of Seti so have gone thru alot of Projects over the years. Here's another listing https://stats.free-dc.org/cpidtagb.php?cpid=da0c795e81490ce47752d88cb0cbd5be&theme=4&cols=1
I'm not sure EVERY Project I've ever crunched for is listed but it's enough to get the idea.
To get the most out of your
)
To get the most out of your GPU tasks you should reduce the number of CPU tasks that are running at the same time. Each Nvidia GPU task requires a full CPU core to support the GPU. By reducing the number of CPU tasks may give you a faster times on GPU tasks.
Harri Liljeroos wrote: To
)
And if the CPU is not over committed it will run the CPU tasks faster (in general).
A Proud member of the O.F.A. (Old Farts Association). Be well, do good work, and keep in touch.® (Garrison Keillor)
Hello again! I've noticed
)
Hello again! I've noticed something very strange to me, maybe someone can help me out. I've tried both 0.5 and 0.33 for my GPU utilization, and time to crunch seems to go up linearly with number of jobs going at once. It takes ~600 seconds with one, ~1200 with two, and now ~1800 with three. That is not the behavior I would expect because my GPU is not maxed out on memory bandwidth or processing capacity according to Afterburner.
I have Boinc set to use 20 of my 24 cores, so I should never get more tasks than I have cores. Should I be changing something to set a full 1.0 core for each task rather than the 0.9 that is scheduled by default? Is the gravitational wave search not designed to effectively utilize a GPU when ran in parallel, or is there something else strange going on at my end?