We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months.
So does this mean no more CPU-only grav. wave tasks forthcoming, ever?
Thanks. Working now. Will use proper location in the future.
For the benefit of other readers, was the problem simply due to missing OpenCL libraries?
I took a look at your completed results. The MeerKAT tasks seem to be taking about an hour and you have more than 500 in progress. You also have Gamma Ray Pulsar tasks taking about 19 minutes with around 140 in progress. That's about 550 hours of work - 275 hours per gpu - around 11.5 days in total. Do you really need to have that much work on hand??? What setting do you have for work cache size?
Maybe you're running multiple tasks per GPU although the GRP times seem about right for running just singles. I'm not sure of the expected time for MeerKAT.
If you are running singles, you are going to have problems meeting deadlines with those task numbers. Until you get a proper handle on crunch times, you should set something like 0.5days max for work cache size.
Please start a new thread with your response in 'Problems' and I'll transfer the existing discussion to that thread to get it out of this O3ASHF1 tech news thread.
How much memory should the application use?
Currently i see 2 GPU WUs running (set up that way years ago) and vram maxed out at 8 GB.
If they indeed use 4GB or more per WU a hint of that non-trivial requirements somewhere (at the application choice for example) would be nice. In the last years VRAM requirements didn't matter for current cards, so maybe i'm not the only one who stopped thinking about it.
these tasks need just under 4GB. on my linux host, it shows as 3906MB for the GW task itself. when you add in the overhead for the window manager and desktop graphics, it's just over 4GB.
so a 4GB GPU is not enough to run one task. but if you have an 8GB GPU you should be able to run 2 tasks. running two tasks on my 2080Ti it's using 7935MB.
on a 2080ti, the task takes about 7-8minutes for the main GPU computation portion, and another 8 mins or so for the followup CPU only portion. ~16mins total. but if you run two tasks at a time and stagger them they basically run the same overall speed each, giving you twice the productivity. just make sure you watch CPU usage and allocate CPU in BOINC for it (1.5 threads or so).
We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months.
So does this mean no more CPU-only grav. wave tasks forthcoming, ever?
In the multi-directional searches there are usually a few setups that we can and/or must do on the CPUs, but the all-sky searches run on the GPUs only nowadays.
But there are still ~150k WUs from the old O3MD1V2 (CPU) search that need to be re-ran again, I just didn't find the time yet to dig out which exactly. These should be issued in the next few weeks, under the same O3MD1 label.
Thanks. Working now. Will
)
Thanks. Working now. Will use proper location in the future.
Bernd Machenschalk wrote:We
)
So does this mean no more CPU-only grav. wave tasks forthcoming, ever?
Allen wrote:Thanks. Working
)
For the benefit of other readers, was the problem simply due to missing OpenCL libraries?
I took a look at your completed results. The MeerKAT tasks seem to be taking about an hour and you have more than 500 in progress. You also have Gamma Ray Pulsar tasks taking about 19 minutes with around 140 in progress. That's about 550 hours of work - 275 hours per gpu - around 11.5 days in total. Do you really need to have that much work on hand??? What setting do you have for work cache size?
Maybe you're running multiple tasks per GPU although the GRP times seem about right for running just singles. I'm not sure of the expected time for MeerKAT.
If you are running singles, you are going to have problems meeting deadlines with those task numbers. Until you get a proper handle on crunch times, you should set something like 0.5days max for work cache size.
Please start a new thread with your response in 'Problems' and I'll transfer the existing discussion to that thread to get it out of this O3ASHF1 tech news thread.
Cheers,
Gary.
How much memory should the
)
How much memory should the application use?
Currently i see 2 GPU WUs running (set up that way years ago) and vram maxed out at 8 GB.
If they indeed use 4GB or more per WU a hint of that non-trivial requirements somewhere (at the application choice for example) would be nice. In the last years VRAM requirements didn't matter for current cards, so maybe i'm not the only one who stopped thinking about it.
these tasks need just under
)
these tasks need just under 4GB. on my linux host, it shows as 3906MB for the GW task itself. when you add in the overhead for the window manager and desktop graphics, it's just over 4GB.
so a 4GB GPU is not enough to run one task. but if you have an 8GB GPU you should be able to run 2 tasks. running two tasks on my 2080Ti it's using 7935MB.
on a 2080ti, the task takes about 7-8minutes for the main GPU computation portion, and another 8 mins or so for the followup CPU only portion. ~16mins total. but if you run two tasks at a time and stagger them they basically run the same overall speed each, giving you twice the productivity. just make sure you watch CPU usage and allocate CPU in BOINC for it (1.5 threads or so).
_________________________________________________________________________
Ok, then it's working as
)
Ok, then it's working as intended. So only one GW Task for me so that Video Players and simpler Games don't get problems running simultaneously.
There really should be (V)RAM requirements listet for the applications.
hadron wrote:Bernd
)
In the multi-directional searches there are usually a few setups that we can and/or must do on the CPUs, but the all-sky searches run on the GPUs only nowadays.
But there are still ~150k WUs from the old O3MD1V2 (CPU) search that need to be re-ran again, I just didn't find the time yet to dig out which exactly. These should be issued in the next few weeks, under the same O3MD1 label.
BM
I received 74 All-Sky
)
I received 74 All-Sky Gravitational Wave - they failed all. Running time about 1:50, then it says (in german) "Berechnungsfehler" and the message:
14.07.2023 22:31:28 | Einstein@Home | BOINC will delete file O3AS1_SegmentList_60h_42nd_removed.seg (no longer needed)
What is the problem?
MyrCu wrote: I received
)
you hosts are hidden. Please link to the affected host directly. Need to see the error messages.
are you running a GPU with 4GB of VRAM or less? If so then that’s likely the problem. These tasks need a tad more than 4GB.
_________________________________________________________________________
OK, thanks - my Nvidia GPU
)
OK, thanks - my Nvidia GPU has only 4 GB.