Discussion Thread for the Continuous GW Search known as O2MD1 (now O2MDF - GPUs only)
Language
Copyright © 2024 Einstein@Home. All rights reserved.
Copyright © 2024 Einstein@Home. All rights reserved.
B.I.G wrote:Seeing it is not
)
I'll try to answer all the points you raise based on my long personal experience in running tasks for this project. I'm a volunteer, just like you, and the opinions I express are my own. I'm NOT speaking in any official capacity on behalf of the project.
I don't know why you should think that the test phase was all about improving efficiency. At the moment, I'm sure those concerns are rather incidental to the main one - to make sure the app gives verifiable answers over a range of different hardware. In other words, the test status has been dropped because the answers are deemed to be 'correct' and in agreement with what the long standing CPU app would produce.
Some problems just can't be 'parallelized' as well as others. There will undoubtedly be ongoing work to attempt to shift more of the work to the GPU. It will take time and unless someone has a really bright idea/new method, the gains will probably be modest rather than spectacular.
Yes, something is terribly "off" here but unfortunately it's the way you are thinking about this.
Let's get a few 'facts' straight, first of all. On AMD GPUs like the RX 580, it does NOT take 1.5 - 2 hrs to crunch a task. It might do so on your system but that's not really something the project (or the app) can control. I know (because I've been measuring it intensely for the last week or more) that an RX 570 can produce a result every 6-9 minutes on a range of different CPU architectures. Even without the benefit of running multiple concurrent GPU tasks, that GPU can still process a task in maybe 15-18 mins or so. You need to 'fix' your system if they take 2 hrs. Maybe you are suffering from some form of severe throttling. I know very little about Macs so wouldn't even try to speculate.
Everybody who has been around for as long as you and I have will already know the oft-quoted mantra of "keep your cache small to start with" :-). Either from bitter past experience or from repeated exhortations in the forums, we've all seen that estimates can be wildly wrong to start with.
Einstein needs locality scheduling for GW searches. Einstein uses older versions of server code to allow that to happen efficiently. The 'penalty' is that duration correction factor (DCF) goes with the older code. There is only one DCF which has to do the job for multiple independent searches. It would be a huge job to implement a '1 DCF per search' mode of operation. It's often requested but it's never going to happen. It's very simple not to get caught. Before you change which searches you are running, reduce your work cache size. 600+ tasks at 8 mins per task is about 3.5 days worth, on top of what you might have had already. You are crazy to change or add a different search with that sort of extra work cache size.
Unfortunately, you are making another set of incorrect assertions. For O2MD searches, CPU tasks and GPU tasks are identical in work content, which is why they have the same credit allocation. The FGRPB1G GPU task is 5x the work content of a standard FGRP CPU task. This is why the GPU task gets 5x693=3465 credits. The GW CPU task gets 1000 because it is very roughly about 1.4 times the work content of a FGRP CPU task. The apparent high credit for the FGRPB1G task simply arises from how well those tasks suit the parallel processing capability of GPUs.
For the current GW GPU tasks, I have a 6C/12T Ryzen 2600 that I just built that is doing ~250 tasks per day on a single RX 570. There are no CPU tasks currently and the machine is running nice and cool. If I took out the GPU and used all 12 threads to crunch CPU tasks, each would take around 10 hrs or so and the machine could produce say ~30 tasks per day. I'm sure the machine would run a lot hotter than it currently is. I would probably be using as much if not more power and producing a hell of a lot less output. My 'saving' would be the $US135 I paid for the GPU. So please have a bit of a rethink about, "I don't see the point in running GW tasks on the GPU at all."
I'm very sorry that it's not working for you but please don't bag the RX 570/580 series of GPUs based on what must surely be a problem with your local setup.
Cheers,
Gary.
This application is not
)
This application is not listed on the official Einstein application webpage:
https://einsteinathome.org/apps.php
Please add by whomever maintains that page.
Thanks.
Darrell wrote:This
)
The very top block of apps showing on that page are the current versions for the GW GPU search. What I'm referring to is headed, "Gravitational Wave search O2 Multi-Directional GPU". There are entries for the 3 OS types as well as both AND and nVidia GPUs. Why do you think they are "not listed"?
Cheers,
Gary.
Thanks for your reply and
)
Thanks for your reply and explanations.
I know the Rx 580 is a great GPU it works awesomely well with the FRGP Tasks and my othe work applications I need for work. And yes it might be a problem with macOS and the GW app. What I am saying at this point it is better not to send any GW GPU tasks to macOS machines because the CPUs crunch them more efficient while the GPUs can do multiple times the calculation with the FGRP tasks.
I monitor the GPU load and it's 50% for the GW task yet it takes that long, so my 10 year old 6 Core Xeon CPU is in fact faster than the otherwise exzellent AMD card for GW tasks. If they work better on Windows / Linux great they should send them out to these systems and let macOS stick to FGRP GPU tasks and do the GW tasks on the CPU. Unfortunately I can't even request GW CPU tasks now because if I allow any GW app for my system I always get GPU tasks, only if I unchek all of them, including the CPU version, the GPU sticks to FGRP tasks.
As for the Scheduler having little buffer, this doesn't work for this machine as it's sometimes offline for 2-3 days when I'm away and of course I want it to do something in that time. But thanks for explaining now I understand how this happens. It uses the reference time from other systems and then get's a big surprise.
I was wondering what are the
)
I was wondering what are the 'lowest' possible GPU models from both AMD and Nvidia that will work with this current GW GPU app.
AMD cards with any chip up to Pitcairn have been seen to be incompatible. These would be cards like HD 7850/7870, R9 270(X) and R9 370(X).
Robert Meckley mentioned elsewhere that AMD R9 280X didn't work. Symptoms and the error had been the same as for Pitcairn. That 280X has Tahiti XTL chip and 280 would have Tahiti Pro.
R9 290 has Hawaii Pro. I would claim that it should work, because R9 390 works and it has that same chip. 290X and 390X would have Hawaii XT. I assume they would work too, unless they have fried (I'm pretty sure most of them have by now... ).
Then again AMD 285 has Tonga Pro and 285X has Tonga XT. Other cards with Tonga chip would be R9 380 and 380X. Does somebody know if cards with Tonga chip are compatible ? That's an interesting border area, because Hawaii is older than Tonga.
Then RX 470/480 have Ellesmere. I don't recall seeing results, but I have a positive feeling those cards could be compatible. How about RX 460, RX 455 and R5 4xx models ? Those have different chips.
Models from Radeon 520 to Radeon RX 560 ? Has anybody tried these ?
Nvidia...
I recall 700-series doesn't work... everything from Kepler is incompatible ?
Maxwell 2.0 cards work well. That would mean GTX 950 minimum and everything up from that in model numbers should work.
But how about GTX 750 (Ti) ? It has Maxwell 'first generation' chip.
Notebook GPUs would be a whole league on their own. I don't have even faint ideas where the dividing lines would be amongst them.
I saw a similar set of
)
I saw a similar set of circumstances when these work units first came out on one of my two hosts, and it is not Mac. My host with a Nvidia 1080 and i7-7820x was crunching the tasks quickly, but my Nvidia 1070 host (with a 6 core i5-8400) was working abysmally slow -- similar to the times of B.I.G. (Both OS are Linux Mint). However, I noticed that I had all six cores on the 1070 host crunching BOINC projects when I forgot to reset it after an experiment back to my normal 5 cores. It made a huge difference 'freeing up' a core! The WU times came way down.
These WUs seem to require significant CPU usage as well. When all six cores on my machine were crunching plus sharing normal system ops, things slowed down.
Since I don't know your setup, just wanted to throw another idea out there for consideration.
Richie wrote:Notebook GPUs
)
The beta tasks worked fine on my MacBook Pro from 2012 with a GT 650m, not as efficient as the FGRP tasks but the difference was not as huge as with the AMD card in the MacPro.
Yes, and not only that, at least on my quite old Xeon system I suspect memory bandwith is a bigger issue than CPU usage. When I use more than 3 cores on my 6 core Westmere Xeon the GPU performance goes down rapidly, with 3 cores the GPU is 30 seconds slower per FGRP task that is enough that the CPU can't compensate so I restricted CPU tasks to 2 cores only.
B.I.G wrote:The beta tasks
)
That sounds like a positively strange exception. GT 650M is basically identical to GTX 650. Both are 'Kepler' with GK107 chip, differences only in clock speeds and memory configuration (DDR3/GDDR5). I recall seeing at least a couple of GTX 650's showing typical incompatibility symptoms ("time limit exeeded" error). Maybe it's also possible that some GPU model could work in Mac even if it doesn't work under linux or Windows, and vice versa.
B.I.G wrote:Thanks for your
)
Pause the CPU work and see what happens to your GPU util and thus run times. Mine were in the dumpster with single digit util with a single task. (Linux/NV) Pausing BOINC CPU tasks changed it to where it should be. The exe priority seems too low for GPU tasks.
Unfortunately no chance at
)
Unfortunately no change at all, before I was running GW tasks with only 2 cores crunching CPU tasks. Now trying one GW task with only the GPU running and exactly the same behaviour: GPU is at 50% load, CPU at 5% and it would take ages to finish, but going to abort this as there is no use to wait.
Maybe it requires macOS 10.14 or even 10.15? This machine is still running 10.13. I don't know... hope the devs manage to fix this. I'd love to crunch GW tasks.