All-Sky Gravitational Wave Search on O3 data (O3ASHF1)

Forums › Technical News

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4332
Credit: 251659453
RAC: 35567
15 Jun 2023 10:21:16 UTC
Topic 229667
(moderation:
    )

    We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months.

    To reach a better sensitivity, we'll make the resulting list of candidates even longer. This means not only larger result files to be uploaded (well, shouldn't be much of a problem nowadays), but also a longer time taken for the "recalc" step ("recalculating toplist ststistics" is written in stderr). This step is done purely on the CPU. We are working on porting it to the GPUs, but the memory access pattern of this step is so unpredictable that we don't get much speedup from that yet (accessing "global" memory on the GPU is still terribly slow). We hope to get an improved version of the App out during the run.

    For the next two weeks there will only be few workunits issued for testing, the real run will start in July.

    Happy crunching!

    BM

    mikey
    mikey
    Joined: 22 Jan 05
    Posts: 12780
    Credit: 1868210749
    RAC: 1862069

    Bernd Machenschalk wrote:We

    15 Jun 2023 14:14:20 UTC
    Message 213575
    (moderation:
      )

      Bernd Machenschalk wrote:
      We are starting a new search for GWs on O3 data, this time going to higher frequencies than in the first all-sky search. This will be a GPU-only search. It is targeted to run for 7-8 months. To reach a better sensitivity, we'll make the resulting list of candidates even longer. This means not only larger result files to be uploaded (well, shouldn't be much of a problem nowadays), but also a longer time taken for the "recalc" step ("recalculating toplist ststistics" is written in stderr). This step is done purely on the CPU. We are working on porting it to the GPUs, but the memory access pattern of this step is so unpredictable that we don't get much speedup from that yet (accessing "global" memory on the GPU is still terribly slow). We hope to get an improved version of the App out during the run. For the next two weeks there will only be few workunits issued for testing, the real run will start in July. Happy crunching!

      So besides the "Run test applications" what do I click to run these and not any other gpu app?

      DF1DX
      DF1DX
      Joined: 14 Aug 10
      Posts: 106
      Credit: 3904303640
      RAC: 598413

      Runtime Error:Stderr

      15 Jun 2023 15:38:22 UTC
      Message 213579
      (moderation:
        )

        Runtime Error:

        [...]

        einstein_O3AS_1.02_windows_x86_64__GW-opencl-nvidia.exe: unrecognized option `--outputNLoudestRecalcCand=250000'

        [...]

        Ben Scott
        Ben Scott
        Joined: 30 Mar 20
        Posts: 54
        Credit: 1710790379
        RAC: 3522328

        Computation error: Task

        15 Jun 2023 23:49:29 UTC
        Message 213598
        (moderation:
          ) 

          Computation error:

          Task 1484856939

          einstein_O3AS_1.02_x86_64-pc-linux-gnu__GW-opencl-nvidia: unrecognized option `--outputNLoudestRecalcCand=250000'


          DF1DX
          DF1DX
          Joined: 14 Aug 10
          Posts: 106
          Credit: 3904303640
          RAC: 598413

          Upload

          16 Jun 2023 11:18:09 UTC
          Message 213619
          (moderation:
            )

            Upload Error:

            [...]
</stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>h1_0813.80_O3aC01Cl1In0__O3ASHF1_814.00Hz_17896_5_0</file_name>
  <error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>

             

            Bernd Machenschalk
            Bernd Machenschalk
            Moderator
            Administrator
            Joined: 15 Oct 04
            Posts: 4332
            Credit: 251659453
            RAC: 35567

            Thanks. Yes, output file size

            16 Jun 2023 11:30:34 UTC
            Message 213621
            (moderation:
              )

              Thanks. Yes, output file size was specified too small. Fixed for newly generated tasks, sorry, I can't do anything about the ones already sent. You may abort the uploads or tasks.

              BM

              DF1DX
              DF1DX
              Joined: 14 Aug 10
              Posts: 106
              Credit: 3904303640
              RAC: 598413

              i now have three computed

              16 Jun 2023 15:21:22 UTC
              Message 213636
              (moderation:
                )

                i now have three computed GW-Wus where the upload is backing off after a few seconds.

                My Host.

                A scheduler-request lasts also with 3 minutes IMHO unusually long.

                Fr 16 Jun 2023 17:10:00 CEST | Einstein@Home | Started upload of h1_0871.80_O3aC01Cl1In0__O3ASHF1_872.00Hz_20536_9_0
                Fr 16 Jun 2023 17:10:00 CEST | Einstein@Home | Started upload of h1_1067.80_O3aC01Cl1In0__O3ASHF1_1068.00Hz_30806_1_0
                Fr 16 Jun 2023 17:10:04 CEST | Einstein@Home | Backing off 00:41:06 on upload of h1_0871.80_O3aC01Cl1In0__O3ASHF1_872.00Hz_20536_9_0
                Fr 16 Jun 2023 17:10:04 CEST | Einstein@Home | Backing off 00:28:43 on upload of h1_1067.80_O3aC01Cl1In0__O3ASHF1_1068.00Hz_30806_1_0

                 

                 

                Bernd Machenschalk
                Bernd Machenschalk
                Moderator
                Administrator
                Joined: 15 Oct 04
                Posts: 4332
                Credit: 251659453
                RAC: 35567

                I'm afraid there are a few

                16 Jun 2023 16:25:00 UTC
                Message 213640
                (moderation:
                  )

                  I'm afraid there are a few things still wrong with that setup. Also the result file size is much larger than what we anticipated. I'll suspend GW Apps for two weeks while we are investigating and fixing.

                  BM

                  magic_sam
                  magic_sam
                  Joined: 30 Dec 21
                  Posts: 23
                  Credit: 556699183
                  RAC: 566

                  Hi all, I second DF1DX's

                  16 Jun 2023 19:54:21 UTC
                  Message 213651
                  (moderation:
                    )

                    Hi all,

                    I second DF1DX's report, tasks are failing with the following error message on Ubuntu 22.04 / AMD Radeon RX 7900 XTX:

                    </stderr_txt>
<message>
upload failure: <file_xfer_error>
  <file_name>h1_0847.80_O3aC01Cl1In0__O3ASHF1_848.00Hz_19421_6_0</file_name>
  <error_code>-131 (file size too big)</error_code>
</file_xfer_error>
</message>
]]>

                    https://einsteinathome.org/fr/task/1485056425

                    https://einsteinathome.org/fr/task/1485169803

                    Looking forward to crunching O3AS :)

                    Cheers, Sam

                    Allen
                    Allen
                    Joined: 23 Jan 06
                    Posts: 75
                    Credit: 691220144
                    RAC: 1019734

                    Hello: Just trying to get

                    20 Jun 2023 19:56:45 UTC
                    Message 213849
                    (moderation:
                      )

                      Hello:

                      Just trying to get setup hear.  Having some trouble getting the GPU apps to run.  I have all the apps checked except All-Sky Gravitational Wave search on O3 (O3AS) and Multi-Directional Gravitational Wave search on O3 (CPU) (O3MD1) and am not getting any work.  Project sees my two RX560 and says

                      6/20/2023 3:54:51 PM | Einstein@Home | Not requesting tasks: don't need (CPU: job cache full; AMD/ATI GPU: )

                      Since the GPUs are idle, something is not right.  Help!

                       

                      Thanks,  Allen

                      Gary Roberts
                      Gary Roberts
                      Moderator
                      Joined: 9 Feb 05
                      Posts: 5874
                      Credit: 118375995480
                      RAC: 25558732

                      Allen wrote:...  Having some

                      20 Jun 2023 23:06:00 UTC
                      Message 213860 in response to message 213849
                      (moderation:
                        )

                        Allen wrote:

                        ...  Having some trouble getting the GPU apps to run.

                        ...  Project sees my two RX560 and says ...

                        Yes, BOINC detects your RX 560 GPUs but do you actually have the required OpenCL libraries installed?  You're running Windows 7 so those libraries aren't automatically installed (I believe from what others have mentioned - I don't run Windows).  Unless you separately installed OpenCL, that may be the problem.

                        The easiest way to know that is to look for a particular line in the BOINC startup messages that you can see in the BOINC Manager event log or by browsing the stdoutdae.txt file in the BOINC folder if you haven't restarted BOINC recently.  Here is an example (from one of my Linux machines) for the sort of thing you should find if OpenCL is correctly installed:-

                        OpenCL: AMD/ATI GPU 0: Radeon RX 570 Series (driver version 3180.7, device version OpenCL 1.2 AMD-APP (3180.7), 3983MB, 3983MB available, 5095 GFLOPS peak

                        You should find two lines (for GPU0 and GPU1).  If you don't have lines like that, you need to install OpenCL.

                        As a general note, the 'Tech News' forum is for announcements from the Devs, eg. about new searches.  To avoid clutter, user responses should only be about problems specific to the announcement.  Your problem should really be in either 'Getting Started' or 'Problems'.  If you need to ask followup questions, please start a new thread in one of those places.

                        Cheers,
                        Gary.

                        Comment viewing options

                        Select your preferred way to display the comments and click "Save settings" to activate your changes.

                        Forums › Technical News