If you just look at a single result, you could work out your problem.
A result at random shows run time of 21,254 with a CPU time of just 44 secs. GPU tasks are being completely starved of CPU support.
Your host shows as 4 cores. Is that 4C/4T or 2C/4T? In either case (it doesn't really matter) it looks like you are running CPU tasks on all available threads. For 27th Sep, I counted 11 CPU tasks returned with that particular return date. The average time for a CPU task is ~34.5 ksecs. So a rough ball park calculation says that 11 tasks would consume ~380 ksecs. A full day is 86.4 ksecs so the number of active cores is 380/86.4 > 4 and that is probably the problem.
Just change your prefs for that machine to allow BOINC to use 75% of the CPUs rather than all of them and I'm guessing the GPU tasks will speed up enormously.
Update and new problems with this host.
Indeed, after limiting CPU load to 75% (1 core is free from calculating CPU only tasks) GPU tasks were calculated much faster. Binary Radio Pulsar Search (MeerKAT) v0.12 () windows_x86_64 calculation time improved from ~ 20,000 s to ~ 1200 s, but 100% fail with “Validate error” or “Error while computing”.
So I had to disable GPU calculations on this computer until application is updated.
It looks to me that the validation rates, in particular of the Linux 0.17 app version, got a lot better with time, without us changing anything on the project side. I see <1.5% invalids of this app version. Does anyone know why this is? Are Linux people with a lot of invalids just shying away from BRP7?
all of my invalids have come down to ~3% or so. Myself and everyone on my team (mostly Linux users with Petri's app) have seen the same trend that validations have gotten a lot better. looks to be the same story with everyone on the leaderboard, including many linux users of both AMD and Nvidia.
Really? cuda55 for Windows and cuda102 for Linux. Why are Windows users getting dealt the lower hand with these applications?
Some of us can not run Linux and need Windows only. Apps should be equal across platforms.
We did a comparison between CUDA 5.5 and CUDA 11, and found that for our application the performance gain of course depends on the card, but is always is <8%. Our boundary is 10% averaged across all cards, below that we judge the benefit noth worth the additional effort in building, deploying and maintaining another app version. So for Windows we went with the more compatible CUDA version, to harvest the computing power of even older cards (of which we do have a few on our own).
CUDA on Linux is a tricky thing, as the pre-compiled shared libraries from NVidia link to a specific version of libgcc, and thus must be linked with that version of gcc. However, for our application the gcc version the CPU code is compiled with is pretty peculiar to ensure good validation. I feel pretty lucky to have found a combination of OS, CUDA libs cnd gcc version that works reasonably well for us.
CUDA on Linux is a tricky thing, as the pre-compiled shared libraries from NVidia link to a specific version of libgcc, and thus must be linked with that version of gcc.
I know you said this before, but I'm not sure I understand exactly what you mean about specific versions "must" be used. I never had any issue building any CUDA apps with seemingly any version of GCC.
our custom app currently is built with the latest CUDA 12.2 Update 2. Normally I build in my Ubuntu 18.04 VM, which by default has gcc 7.5.0, I'be built everything from CUDA 9 to CUDA12 apps on this environment. I only maintain this old 18.04 environment so that i don't have to deal with building on a latest release and someone trying to use the app on an older OS and getting locked out by the glibc dependency. After your comments about using 7.3 for better validation, I simply installed gcc 7.3.0 into that VM (using a script I found on github), then set some env vars to bring that version of gcc into my build environment (terminal session). the app built just as fine, and does indeed seem to validate better. and the binaries were definitely different between 7.5.0 and 7.3.0 builds.
I can't complain because it's working well now. I just don't understand what you mean that certain versions of gcc "must" be used as that doesnt seem to be the case. maybe certain versions work better than others for validation, but there's nothing stopping the build from succeeding in my experience.
Thanks. The Ubuntu 18.04 is the machine I built this App on, I'll try building an official App with gcc 7.3 there, thanks!
What I said was based on my experience trying to build a CUDA 5.5 app for Linux, which simply didn't work with a gcc >4. But apparently NVidia removed that dependency to libgcc in newer versions of CUDA.
ah, you could be right about that with older cuda versions. i did try to build an old cuda version once (cuda 8.0 i think) and couldnt ever get it to work. building older cuda apps seems like it was much more of a bear.
the cuda 10.2 app you already have up for linux beta worked well too when i tested it.
Gary Roberts
)
Update and new problems with this host.
Indeed, after limiting CPU load to 75% (1 core is free from calculating CPU only tasks) GPU tasks were calculated much faster. Binary Radio Pulsar Search (MeerKAT) v0.12 () windows_x86_64 calculation time improved from ~ 20,000 s to ~ 1200 s, but 100% fail with “Validate error” or “Error while computing”.
So I had to disable GPU calculations on this computer until application is updated.
Really? cuda55 for Windows
)
Really? cuda55 for Windows and cuda102 for Linux. Why are Windows users getting dealt the lower hand with these applications?
Some of us can not run Linux and need Windows only. Apps should be equal across platforms.
the speed of cuda55 on
)
the speed of cuda55 on windows is the same as the cuda102 on linux as far as i can tell.
but cuda55 on windows validates much better.
not sure what's to complain about that there.
_________________________________________________________________________
Ian&Steve C. wrote:the
)
Haven't ran any yet because I was going off of how crappy the old cuda runs on Moo compared to newer versions of cuda.
Good to know the performance is the same between them here then. Thanks.
It looks to me that the
)
It looks to me that the validation rates, in particular of the Linux 0.17 app version, got a lot better with time, without us changing anything on the project side. I see <1.5% invalids of this app version. Does anyone know why this is? Are Linux people with a lot of invalids just shying away from BRP7?
BM
all of my invalids have come
)
all of my invalids have come down to ~3% or so. Myself and everyone on my team (mostly Linux users with Petri's app) have seen the same trend that validations have gotten a lot better. looks to be the same story with everyone on the leaderboard, including many linux users of both AMD and Nvidia.
_________________________________________________________________________
bluestang wrote:Really?
)
We did a comparison between CUDA 5.5 and CUDA 11, and found that for our application the performance gain of course depends on the card, but is always is <8%. Our boundary is 10% averaged across all cards, below that we judge the benefit noth worth the additional effort in building, deploying and maintaining another app version. So for Windows we went with the more compatible CUDA version, to harvest the computing power of even older cards (of which we do have a few on our own).
CUDA on Linux is a tricky thing, as the pre-compiled shared libraries from NVidia link to a specific version of libgcc, and thus must be linked with that version of gcc. However, for our application the gcc version the CPU code is compiled with is pretty peculiar to ensure good validation. I feel pretty lucky to have found a combination of OS, CUDA libs cnd gcc version that works reasonably well for us.
BM
Bernd Machenschalk
)
I know you said this before, but I'm not sure I understand exactly what you mean about specific versions "must" be used. I never had any issue building any CUDA apps with seemingly any version of GCC.
our custom app currently is built with the latest CUDA 12.2 Update 2. Normally I build in my Ubuntu 18.04 VM, which by default has gcc 7.5.0, I'be built everything from CUDA 9 to CUDA12 apps on this environment. I only maintain this old 18.04 environment so that i don't have to deal with building on a latest release and someone trying to use the app on an older OS and getting locked out by the glibc dependency. After your comments about using 7.3 for better validation, I simply installed gcc 7.3.0 into that VM (using a script I found on github), then set some env vars to bring that version of gcc into my build environment (terminal session). the app built just as fine, and does indeed seem to validate better. and the binaries were definitely different between 7.5.0 and 7.3.0 builds.
I can't complain because it's working well now. I just don't understand what you mean that certain versions of gcc "must" be used as that doesnt seem to be the case. maybe certain versions work better than others for validation, but there's nothing stopping the build from succeeding in my experience.
_________________________________________________________________________
Thanks. The Ubuntu 18.04 is
)
Thanks. The Ubuntu 18.04 is the machine I built this App on, I'll try building an official App with gcc 7.3 there, thanks!
What I said was based on my experience trying to build a CUDA 5.5 app for Linux, which simply didn't work with a gcc >4. But apparently NVidia removed that dependency to libgcc in newer versions of CUDA.
Thanks, I'll try that again.
BM
ah, you could be right about
)
ah, you could be right about that with older cuda versions. i did try to build an old cuda version once (cuda 8.0 i think) and couldnt ever get it to work. building older cuda apps seems like it was much more of a bear.
the cuda 10.2 app you already have up for linux beta worked well too when i tested it.
_________________________________________________________________________