The task gets stuck at 89.997% for a very very long time

Forums › Problems and Bug Reports

Eliovich Alexander & Yan
Eliovich Alexan...
Joined: 12 Mar 20
Posts: 22
Credit: 475782741
RAC: 104014
10 Jan 2022 16:00:33 UTC
Topic 226742
(moderation:
    )

    The computation task on the video card gets stuck at 89.997% for a very very long time.

    If my patience is not exhausted and I have not interrupted this disgrace, after a few days it may report a calculation error.

     

    Details:

    Gamma-rai Pulsar Search, Windows 7 64 bit Professional. 8 gigabytes of memory and part of it is free. More than enough disk space. 2 of 4 processor cores work and one more with a GPU, one core is always free , Boinc manager 7.16.20,

    Dell computer OptiPlex 7450 with Intel HD Graphics 630.  Drivers for Win 7 taken from HD 620 and in all other cases work fine (tested by a large number of users).

    I don't think the problem is with the drivers. There were no problems when processing gravitational waves on this video card.

    According to the forum, ~ 90% stuck issue on a wide variety of platforms has been noted more than once.

    Eliovich Alexander & Yan
    Eliovich Alexan...
    Joined: 12 Mar 20
    Posts: 22
    Credit: 475782741
    RAC: 104014

    It writes in the report that

    10 Jan 2022 16:16:19 UTC
    Message 191675
    (moderation:
      )

      It writes in the report that there is not enough memory, but this is not true.

       

      17:36:02 (3232): [normal]: Start of BOINC application 'projects/einstein.phys.uwm.edu/hsgamma_FGRPB1G_1.22_windows_x86_64__FGRPopencl-intel_gpu.exe'.
17:36:02 (3232): [debug]: 1e+016 fp, 4.9e+009 fp/s, 2137550 s, 593h45m49s77
17:36:02 (3232): [normal]: % CPU usage: 1.000000, GPU usage: 1.000000
command line: projects/einstein.phys.uwm.edu/hsgamma_FGRPB1G_1.22_windows_x86_64__FGRPopencl-intel_gpu.exe --inputfile ../../projects/einstein.phys.uwm.edu/LATeah3011L06.dat --alpha 2.59819959601 --delta -0.694603692878 --skyRadius 1.890770e-06 --ldiBins 15 --f0start 628.0 --f0Band 8.0 --firstSkyPoint 0 --numSkyPoints 1 --f1dot -1e-13 --f1dotBand 1e-13 --df1dot 1.69860773e-15 --ephemdir ..\..\projects\einstein.phys.uwm.edu\JPLEPH --Tcoh 2097152.0 --toplist 10 --cohFollow 10 --numCells 1 --useWeights 1 --Srefinement 1 --CohSkyRef 1 --cohfullskybox 1 --mmfu 0.1 --reftime 56100 --model 0 --f0orbit 0.005 --mismatch 0.1 --demodbinary 1 --BinaryPointFile ../../projects/einstein.phys.uwm.edu/templates_LATeah3011L06_0636_17864523.dat --debug 0 --device 0 -o LATeah3011L06_636.0_0_0.0_17864523_1_0.out
output files: 'LATeah3011L06_636.0_0_0.0_17864523_1_0.out' '../../projects/einstein.phys.uwm.edu/LATeah3011L06_636.0_0_0.0_17864523_1_0' 'LATeah3011L06_636.0_0_0.0_17864523_1_0.out.cohfu' '../../projects/einstein.phys.uwm.edu/LATeah3011L06_636.0_0_0.0_17864523_1_1'
17:36:02 (3232): [debug]: Flags: X64 SSE SSE2 GNUC X86 GNUX86
17:36:02 (3232): [debug]: Set up communication with graphics process.
boinc_get_opencl_ids returned [0000000001369200 , 0000000001335d40] 
Using OpenCL platform provided by: Intel(R) Corporation
Using OpenCL device "Intel(R) HD Graphics 630" by: Intel(R) Corporation
Max allocation limit: 1360632218
Global mem size: 1360632218
OpenCL device has FP64 support
% checkpoint read: skypoint 7 binarypoint 645
Error in OpenCL context: Out of device memory.
Error in OpenCL context: Out of resources.
Error during OpenCL host->device transfer read coh_followup_list(error: -5)
17:36:34 (3232): [CRITICAL]: ERROR: MAIN() returned with error '1'
FPU status flags:  COND_1 PRECISION
17:36:39 (3232): [normal]: done. calling boinc_finish(65).
17:36:39 (3232): called boinc_finish
      Ian&Steve C.
      Ian&Steve C.
      Joined: 19 Jan 20
      Posts: 3958
      Credit: 46991462642
      RAC: 64834535

      Intel iGPUs share system

      10 Jan 2022 16:18:48 UTC
      Message 191676
      (moderation:
        )

        Intel iGPUs share system memory, however they usually are not allocated ALL system memory, usually a set value or percentage.

        check the BIOS if you can allocate more, and also check the BOINC compute settings to allow higher system memory use.

        _________________________________________________________________________

        Eliovich Alexander & Yan
        Eliovich Alexan...
        Joined: 12 Mar 20
        Posts: 22
        Credit: 475782741
        RAC: 104014

        Thanks for the advice!I

        10 Jan 2022 17:35:00 UTC
        Message 191677
        (moderation:
          )

          Thanks for the advice!

          I do not see such a setting in BIOS.

          I set Boink to 80% RAM and paging file usage.

          Now I will increase it to 90% and see if something changes.

           

          P.S.  Nothing has changed, still the same problem

          Ian&Steve C.
          Ian&Steve C.
          Joined: 19 Jan 20
          Posts: 3958
          Credit: 46991462642
          RAC: 64834535

          Check the memory allocation

          10 Jan 2022 18:08:29 UTC
          Message 191682
          (moderation:
            )

            Check the memory allocation for the iGPU in the BIOS, if it’s available. It’s possible you cannot allocate more. 

            _________________________________________________________________________

            Eliovich Alexander & Yan
            Eliovich Alexan...
            Joined: 12 Mar 20
            Posts: 22
            Credit: 475782741
            RAC: 104014

            Alas, I don’t think it’s a

            10 Jan 2022 20:11:57 UTC
            Message 191684
            (moderation:
              )

              1) I do not see anything similar in BIOS

              2) Alas, I don’t think it’s a lack of memory.

              When I counted gravitational waves, memory was consumed more, but there were no problems.

              In addition, I put the calculation on only one core together with the video card, the memory consumption is minimal, but the problem does not disappear.

              The devil knows what the reason is. (((

              Gary Roberts
              Gary Roberts
              Moderator
              Joined: 9 Feb 05
              Posts: 5872
              Credit: 117672185986
              RAC: 35162397

              Eliovich Alexander & Yan

              10 Jan 2022 22:56:19 UTC
              Message 191690 in response to message 191684
              (moderation:
                )

                Eliovich Alexander & Yan wrote:
                ... Alas, I don’t think it’s a lack of memory.

                The output you posted in your second message disagrees with you :-).

                Error in OpenCL context: Out of device memory.

                I notice you are running both CPU tasks and GPU tasks.  Why don't you just temporarily suspend all CPU tasks and just try running single GPU tasks to see if they suddenly complete without issue?  If they do, that would then prove to you that there is truly a genuine memory contention issue.  It's not clear to me from what you have posted that you have previously tried this.

                Over many years, there have been problems trying to use Intel GPUs at Einstein, with all the variations in hardware and differences in drivers.  Some combinations work but many don't.  It's been a real mess and usually is quite difficult to find a combination that works reliably.

                In your first post you noted the long pause at 89.997% completed.  Even discrete GPUs see a delay at 89.997% complete, although its usually only a minute or two in many cases.  It's due to the main computation completing here, with a followup stage in processing the top 10 candidates occupying the region from 90 - 100%  The current series (LATeah3011Lnnn) of tasks have a particularly noticeable followup stage.

                There are no progress markers during the followup stage so you just need to be patient.  The followup stage uses double precision - which is apparently rather slow in the case of an Intel GPU.

                Cheers,
                Gary.

                Eliovich Alexander & Yan
                Eliovich Alexan...
                Joined: 12 Mar 20
                Posts: 22
                Credit: 475782741
                RAC: 104014

                Why don't you just

                11 Jan 2022 2:01:52 UTC
                Message 191693
                (moderation:
                  )

                  Why don't you just temporarily suspend all CPU tasks and just try running single GPU tasks to see if they suddenly complete without issue?"

                  -- yes of course, as I already wrote, I tried this and the result was the same -- freezing at the same percentage

                    There are no progress markers during the followup stage so you just need to be patient.        
                   
                   -- OK, patience is our everything. But I notice that the device manager shows at this stage 0% of the processor's work -- for many hours. It doesn't look like something is actually being calculated.

                  And unfortunately, in most cases, after a very long freeze, the program reports a calculation error.

                  So, it seems to me that something is still wrong here.

                  Thank you for your attention

                  Gandolph1
                  Gandolph1
                  Joined: 20 Feb 05
                  Posts: 180
                  Credit: 389650951
                  RAC: 1117

                  Been a long time since I used

                  11 Jan 2022 13:50:48 UTC
                  Message 191704
                  (moderation:
                    )

                    Been a long time since I used a DEll cant recall the model number but I was in the BIOS many times and there was most definitely a setting for how much main memory to allocate to the IGPU.  Again, long time since I used a machine with integrated graphics but every machine I've ever used and built (Yes I even built a couple for friends that didn't need the discrete GPU) had this setting.  You just gotta look, check every tab in the BIOS interface being sure that you scroll down for items that may be off the screen.  Its there somewhere...

                     

                    Eliovich Alexander & Yan
                    Eliovich Alexan...
                    Joined: 12 Mar 20
                    Posts: 22
                    Credit: 475782741
                    RAC: 104014

                    Thanks for the

                    11 Jan 2022 22:47:37 UTC
                    Message 191710
                    (moderation:
                      )

                      Thanks for the advice.

                      Looked again, no, there is no such setting.

                      Maybe Windows 7 can do this?

                       

                      By the way, according to the dxdiag, 1824 MB are allocated for video memory, which is a lot

                      Gandolph1
                      Gandolph1
                      Joined: 20 Feb 05
                      Posts: 180
                      Credit: 389650951
                      RAC: 1117

                      Looking thru the dell forums

                      11 Jan 2022 22:54:50 UTC
                      Message 191711
                      (moderation:
                        )

                        Looking thru the dell forums searching for your particular problem yields what you already know.  Apparently there is no way to change the main memory allocation of the integrated GPU. Sorry, I'm tapped out.

                         

                         

                         

                        Comment viewing options

                        Select your preferred way to display the comments and click "Save settings" to activate your changes.

                        Forums › Problems and Bug Reports