All-Sky Gravitational Wave Search on O3 data (O3ASHF1)

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4,324
Credit: 251,072,067
RAC: 41,570
Topic 229667

We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months.

To reach a better sensitivity, we'll make the resulting list of candidates even longer. This means not only larger result files to be uploaded (well, shouldn't be much of a problem nowadays), but also a longer time taken for the "recalc" step ("recalculating toplist ststistics" is written in stderr). This step is done purely on the CPU. We are working on porting it to the GPUs, but the memory access pattern of this step is so unpredictable that we don't get much speedup from that yet (accessing "global" memory on the GPU is still terribly slow). We hope to get an improved version of the App out during the run.

For the next two weeks there will only be few workunits issued for testing, the real run will start in July.

Happy crunching!

BM

mikey
mikey
Joined: 22 Jan 05
Posts: 12,737
Credit: 1,839,138,786
RAC: 3,523

Bernd Machenschalk wrote:We

Bernd Machenschalk wrote:
We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months. To reach a better sensitivity, we'll make the resulting list of candidates even longer. This means not only larger result files to be uploaded (well, shouldn't be much of a problem nowadays), but also a longer time taken for the "recalc" step ("recalculating toplist ststistics" is written in stderr). This step is done purely on the CPU. We are working on porting it to the GPUs, but the memory access pattern of this step is so unpredictable that we don't get much speedup from that yet (accessing "global" memory on the GPU is still terribly slow). We hope to get an improved version of the App out during the run. For the next two weeks there will only be few workunits issued for testing, the real run will start in July. Happy crunching!

So besides the "Run test applications" what do I click to run these and not any other gpu app?

DF1DX
DF1DX
Joined: 14 Aug 10
Posts: 106
Credit: 3,902,867,854
RAC: 2,923,233

Runtime Error:Stderr

Runtime Error:

[...]

einstein_O3AS_1.02_windows_x86_64__GW-opencl-nvidia.exe: unrecognized option `--outputNLoudestRecalcCand=250000'

[...]

Ben Scott
Ben Scott
Joined: 30 Mar 20
Posts: 54
Credit: 1,650,310,605
RAC: 3,339,972

Computation error: Task

Computation error:

Task 1484856939

einstein_O3AS_1.02_x86_64-pc-linux-gnu__GW-opencl-nvidia: unrecognized option `--outputNLoudestRecalcCand=250000'


DF1DX
DF1DX
Joined: 14 Aug 10
Posts: 106
Credit: 3,902,867,854
RAC: 2,923,233

Upload

Upload Error:

[...]
</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>h1_0813.80_O3aC01Cl1In0__O3ASHF1_814.00Hz_17896_5_0</file_name>
  <error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>

 

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4,324
Credit: 251,072,067
RAC: 41,570

Thanks. Yes, output file size

Thanks. Yes, output file size was specified too small. Fixed for newly generated tasks, sorry, I can't do anything about the ones already sent. You may abort the uploads or tasks.

BM

DF1DX
DF1DX
Joined: 14 Aug 10
Posts: 106
Credit: 3,902,867,854
RAC: 2,923,233

i now have three computed

i now have three computed GW-Wus where the upload is backing off after a few seconds.

My Host.

A scheduler-request lasts also with 3 minutes IMHO unusually long.

Fr 16 Jun 2023 17:10:00 CEST | Einstein@Home | Started upload of h1_0871.80_O3aC01Cl1In0__O3ASHF1_872.00Hz_20536_9_0
Fr 16 Jun 2023 17:10:00 CEST | Einstein@Home | Started upload of h1_1067.80_O3aC01Cl1In0__O3ASHF1_1068.00Hz_30806_1_0
Fr 16 Jun 2023 17:10:04 CEST | Einstein@Home | Backing off 00:41:06 on upload of h1_0871.80_O3aC01Cl1In0__O3ASHF1_872.00Hz_20536_9_0
Fr 16 Jun 2023 17:10:04 CEST | Einstein@Home | Backing off 00:28:43 on upload of h1_1067.80_O3aC01Cl1In0__O3ASHF1_1068.00Hz_30806_1_0

 

 

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4,324
Credit: 251,072,067
RAC: 41,570

I'm afraid there are a few

I'm afraid there are a few things still wrong with that setup. Also the result file size is much larger than what we anticipated. I'll suspend GW Apps for two weeks while we are investigating and fixing.

BM

magic_sam
magic_sam
Joined: 30 Dec 21
Posts: 23
Credit: 556,699,183
RAC: 2,761

Hi all, I second DF1DX's

Hi all,

I second DF1DX's report, tasks are failing with the following error message on Ubuntu 22.04 / AMD Radeon RX 7900 XTX:

</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>h1_0847.80_O3aC01Cl1In0__O3ASHF1_848.00Hz_19421_6_0</file_name>
  <error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>

https://einsteinathome.org/fr/task/1485056425

https://einsteinathome.org/fr/task/1485169803

Looking forward to crunching O3AS :)

Cheers, Sam

Allen
Allen
Joined: 23 Jan 06
Posts: 75
Credit: 673,314,565
RAC: 1,025,497

Hello: Just trying to get

Hello:

Just trying to get setup hear.  Having some trouble getting the GPU apps to run.  I have all the apps checked except All-Sky Gravitational Wave search on O3 (O3AS) and Multi-Directional Gravitational Wave search on O3 (CPU) (O3MD1) and am not getting any work.  Project sees my two RX560 and says

6/20/2023 3:54:51 PM | Einstein@Home | Not requesting tasks: don't need (CPU: job cache full; AMD/ATI GPU: )

Since the GPUs are idle, something is not right.  Help!

 

Thanks,  Allen

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,873
Credit: 117,966,180,488
RAC: 24,312,995

Allen wrote:...  Having some

Allen wrote:

...  Having some trouble getting the GPU apps to run.

...  Project sees my two RX560 and says ...

Yes, BOINC detects your RX 560 GPUs but do you actually have the required OpenCL libraries installed?  You're running Windows 7 so those libraries aren't automatically installed (I believe from what others have mentioned - I don't run Windows).  Unless you separately installed OpenCL, that may be the problem.

The easiest way to know that is to look for a particular line in the BOINC startup messages that you can see in the BOINC Manager event log or by browsing the stdoutdae.txt file in the BOINC folder if you haven't restarted BOINC recently.  Here is an example (from one of my Linux machines) for the sort of thing you should find if OpenCL is correctly installed:-

OpenCL: AMD/ATI GPU 0: Radeon RX 570 Series (driver version 3180.7, device version OpenCL 1.2 AMD-APP (3180.7), 3983MB, 3983MB available, 5095 GFLOPS peak

You should find two lines (for GPU0 and GPU1).  If you don't have lines like that, you need to install OpenCL.

As a general note, the 'Tech News' forum is for announcements from the Devs, eg. about new searches.  To avoid clutter, user responses should only be about problems specific to the announcement.  Your problem should really be in either 'Getting Started' or 'Problems'.  If you need to ask followup questions, please start a new thread in one of those places.

Cheers,
Gary.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.