Gravitational Wave search O2 Multi-Directional GPU v2.07 (GW-opencl-nvidia) windows_x86_64
Language
Copyright © 2024 Einstein@Home. All rights reserved.
Copyright © 2024 Einstein@Home. All rights reserved.
Ian&Steve C. wrote:All of the
)
That seems to be true in these cases but Bernd said fixed that on the 17th.
https://einsteinathome.org/goto/comment/176722
Only those 4 miscreants have shown up here
When the first reports of
)
When the first reports of tasks (the new type based on the Vela pulsar) were failing, I checked one of my hosts that was running tasks for the previous pulsar at 3x without issue. I tested and found that the new tasks would fail at 3x so I changed to 2x when the new tasks reached the top of the queue.
Last night, before going home, I checked that host and found no problems at 2x and noticed that the tasks at that point were running a lot faster than previously - 28 mins @2x - 14 mins per task - compared to something over an hour (30 mins per task) for the very first Vela tasks run at 2x. This was just a quick glance at a few tasks. The drop from >30 mins to 14 mins was a surprisingly large difference and I wondered if this meant that the memory requirements might now be less. I immediately reinstated 3x to see what would happen.
I allowed several tasks to complete at that setting. There were no problems at 3x and the crunch time was 36 mins - 12 mins per task - so a further increase in output compared to 2x.
The `frequency bins' being analysed were in the range 1347.95Hz to 1348.35Hz (from the task name) and I had lots of consecutive 'sequence numbers' (_250 all the way down to _50) so I wondered if it would be safe to let all these run at 3x. So I took the risk and just went home, leaving it at 3x.
This morning, when I checked, there are zero errors for running at 3x and the current tasks have sequence numbers around _90. The big difference is that current tasks are taking around 90 mins at 3x - back to 30 mins per task.
Seeing as my host seems to be the only one currently "drinking from this particular frequency cup", I might spend a bit of time today analysing all the available consecutively reducing sequence numbers to see if there is any particular correlation between where in the sequence a task lies and how long it takes to crunch. To do that I'll need to consult the tasks list on the website fairly thoroughly. It might take a while :-).
Of course, if errors start to show up I'll go back to 2x, but so far so good. If anything of interest shows up, I'll report it in the O2MDF discussion thread where those interested in the performance of this app will be able to find it more easily.
EDIT: Of course, I should mention that the host is a Ryzen 5 2600 (6c/12T, 8GB) driving an AMD RX 570 4GB and running GPU tasks only :-).
Cheers,
Gary.
your tasks failed with
)
your tasks failed with "Processing FFT failed: CL_MEM_OBJECT_ALLOCATION_FAILURE" not enough memory.
I'm processing several tasks right now which are using >3200MB GPU memory...
also note that you cannot use all 100% of your memory on an nvidia card for opencl jobs. you're limited to like 1/4 of that.
_________________________________________________________________________
On the 17th Bernd said he
)
On the 17th Bernd said he fixed that.
https://einsteinathome.org/goto/comment/176722Yeah you mentioned that in
)
Yeah you mentioned that in your last post, but obviously not ”fixed“ if tasks are still going out requiring so much memory. My screenshot was from today.
_________________________________________________________________________
True, those tasks were
)
True, those tasks were created on the 19th
Gary Roberts wrote:EDIT: Of
)
The good thing about AMD is they use somewhere between 56-67% of Available RAM of the GPU. I think Keith is the one that mentioned, that if Nvidia ever got around to using OpenCL 2.0 then the amount of available ram for the Nvidia cards would go up to 50%. One can only wish.....
Z, what is your opinion on my
)
Z, what is your opinion on my screenshot? 3217MB is about 40% of my available ram on that nvidia 8GB card.
and that's 3200 for that application alone, not for the whole GPU.
_________________________________________________________________________
Hey Ian, Truthfully I've
)
Hey Ian,
Truthfully I've not paid alot of attention to what the GW are using on my systems since we are running high end GPUs with lots of ram. I'm assuming that is nvidia-smi? I usually just use BonicTasks to monitor the work units. Physical memory around 350-452 MB and virtual Memory around 10893-12916 MB. Let me look. Just opened up the nvidia-smi, yup that looks about right 3239MiB per work unit. That looks like the normal amount. They are hunger buggers, lol...
Z
I meant more in reference to
)
I meant more in reference to this 27% limit. im certainly using more than that at 3200. So maybe the limit is different now?
_________________________________________________________________________