Disk space used for unused application

Forums › Problems and Bug Reports

AgentB
AgentB
Joined: 17 Mar 12
Posts: 915
Credit: 513211304
RAC: 0
3 Jul 2017 18:23:18 UTC
Topic 208601
(moderation:
    )

    I noticed an error about boinc needing 100MB for a a  task and i had run out of space (10GB)

    Mon 03 Jul 2017 15:58:55 BST | Einstein@Home | Continuous Gravitational Wave search Galactic Center highFreq needs 84.59MB more disk space.  You currently have 15.41 MB available and it needs 100.00 MB.

    Mon 03 Jul 2017 15:58:55 BST | Einstein@Home | Gamma-ray pulsar binary search #1 needs 3.67MB more disk space.  You currently have 15.41 MB available and it needs 19.07 MB.

    So this seemed a lot and i figured it needed to download an updated application, as the tasks were not that large.

    So a trundled over to look at who were eating all the disk pies...

     

     3832104 Jul 12  2015 einstein_S5R6_1.01_graphics_i686-pc-linux-gnu  6278871 Feb  2  2016 hsgamma_FGRP4_1.15_x86_64-pc-linux-gnu__FGRP4-SSE2  6320021 Feb  2  2016 hsgamma_FGRPB1_1.00_x86_64-pc-linux-gnu  6320021 Sep 13  2016 hsgamma_FGRPB1_1.01_x86_64-pc-linux-gnu__FGRPOLD  6401891 Dec  9  2016 hsgamma_FGRPB1G_1.05_x86_64-pc-linux-gnu__FGRPSSE  6401891 Sep 26  2016 hsgamma_FGRPB1_1.05_x86_64-pc-linux-gnu__FGRPSSE-Beta  6464399 Jun 21 13:09 hsgamma_FGRPB1_1.06_x86_64-pc-linux-gnu__FGRPSSE  8109487 Oct  4  2016 einsteinbinary_BRP4G_1.39_x86_64-pc-linux-gnu__BRP4G-opencl-ati  8123333 Aug 31  2015 einsteinbinary_BRP4G_1.52_x86_64-pc-linux-gnu__BRP4G-Beta-opencl-ati  8123333 Jun  4  2015 einsteinbinary_BRP6_1.53_x86_64-pc-linux-gnu__BRP6-opencl-ati 10199802 Dec 20  2016 hsgamma_FGRPB1G_1.17_x86_64-pc-linux-gnu__FGRPopencl-ati 10216922 Feb  8 10:06 hsgamma_FGRPB1G_1.18_x86_64-pc-linux-gnu__FGRPopencl1K-ati 10216922 Jan 16 09:02 hsgamma_FGRPB1G_1.18_x86_64-pc-linux-gnu__FGRPopencl-Beta-ati 12272855 Aug 31  2015 einsteinbinary_BRP4_1.00_graphics_i686-pc-linux-gnu 23463592 Jul 12  2015 einstein_S6BucketFU2UB_1.01_x86_64-pc-linux-gnu__X64 23463592 Sep  8  2015 einstein_S6BucketFU3UB_1.01_x86_64-pc-linux-gnu__X64 28296618 Feb 14  2016 einstein_O1AS20-100T_1.02_i686-pc-linux-gnu__SSE2 29826080 Feb 13  2016 einstein_O1AS20-100T_1.02_x86_64-pc-linux-gnu 29827040 Feb 16  2016 einstein_O1AS20-100T_1.03_x86_64-pc-linux-gnu__X64 66192587 Oct 25  2016 einstein_O1MD1TCV_1.03_x86_64-pc-linux-gnu__AVX 66197129 Nov 23  2016 einstein_O1MD1CV_1.00_x86_64-pc-linux-gnu__AVX 66205256 Jun  1 11:22 einstein_O1Spot1TLo_1.04_x86_64-pc-linux-gnu__AVX 66205256 Jun  7 13:00 einstein_O1Spot1Hi_1.00_x86_64-pc-linux-gnu__AVX 66205256 May 19 20:20 einstein_O1Spot1THi_1.04_x86_64-pc-linux-gnu__AVX 72444186 Feb 19  2016 einstein_O1AS20-100T_1.04_x86_64-pc-linux-gnu__AVX 72444186 Mar  9  2016 einstein_O1AS20-100F_1.04_x86_64-pc-linux-gnu__AVXO1F

    So can some of these be deleted?

    Does the project have a mechanism for freeing these up ?

     edit: apologies for the formatting - it looks perfectly fine in the edit window (FireFox)!  (press reply to see it formatted correctly)

    Richie
    Richie
    Joined: 7 Mar 14
    Posts: 656
    Credit: 1702989778
    RAC: 0

    I assume those that aren't

    3 Jul 2017 20:04:11 UTC
    Message 159492
    (moderation:
      )

      I assume those that aren't recently mentioned here (app and exact version number) are ancient apps or versions already and are not needed anymore:

      https://einsteinathome.org/apps.php

      https://einsteinathome.org/server_status.html

      https://einsteinathome.org/apps.php?xml=1

      strings... BRP4G, BRP6, S6Bucket, O1AS20, O1MD1CV, O1Spot1T, and every 'Beta'

      I think you could delete all of those. 

      Gary Roberts
      Gary Roberts
      Moderator
      Joined: 9 Feb 05
      Posts: 5872
      Credit: 117754882097
      RAC: 34819835

      Richie_9 wrote:strings...

      4 Jul 2017 0:23:00 UTC
      Message 159499 in response to message 159492
      (moderation:
        )

        Richie_9 wrote:

        strings... BRP4G, BRP6, S6Bucket, O1AS20, O1MD1CV, O1Spot1T, and every 'Beta'

        I think you could delete all of those. 

        If you do, your BOINC client will probably request them all back again.

        This issue comes up from time to time.  I remember Bernd saying something to the effect that it was difficult from the server side to 'unsticky' old apps that were no longer current as there wasn't a suitable BOINC mechanism - or something along those lines - I don't remember the details.

        If I want to clean up old apps, I remove the entries for them from the state file before deleting them on disk.  That way, BOINC 'forgets' about them and doesn't download replacements.  I rarely do this as it's easier just to allocate a bit more disk space that BOINC can use.  I'm not advocating that people should tamper with the state file unless they really know what they are doing.  I just want to make sure people understand that just deleting the file won't solve the problem.

         EDIT:  The 'official' way to retrieve the disk space would be to set NNT and finish off the tasks on hand.  After returning the completed results, resetting the project would get rid of all the old entries.  The 'cost' would be that all 'current' files would need to be downloaded afresh.  This might be quite a lot with the current GW search.  You could avoid most of this if you had taken a backup copy of current files beforehand and restored them before BOINC tried to download them again.

         

        Cheers,
        Gary.

        Christian Beer
        Christian Beer
        Joined: 9 Feb 05
        Posts: 595
        Credit: 188538371
        RAC: 180591

        This is a BOINC wide problem.

        4 Jul 2017 7:46:59 UTC
        Message 159507
        (moderation:
          )

          This is a BOINC wide problem. There is currently no way for a project to tell the Client that applications can be deleted. Other than manually fiddling with client_state, a project reset is the best way after setting no new work. As Gary said this means that all GW related datafiles will also be downloaded again. You could backup the files but that would still involve client_state fiddling.

          AgentB
          AgentB
          Joined: 17 Mar 12
          Posts: 915
          Credit: 513211304
          RAC: 0

          I tried removing a few files

          4 Jul 2017 22:00:00 UTC
          Message 159545 in response to message 159507
          (moderation:
            )

            I tried removing a few files and sure enough on boinc restart it restored them.

            edit : i wish i had found xmlcopyeditor earlier, it does make looking at xml files a little easier.

             

             

             

            Gary Roberts
            Gary Roberts
            Moderator
            Joined: 9 Feb 05
            Posts: 5872
            Credit: 117754882097
            RAC: 34819835

            I just use KDE's kwrite for

            4 Jul 2017 22:50:29 UTC
            Message 159546
            (moderation:
              )

              I just use KDE's kwrite for all shell scripts and BOINC xml file editing.  I find it very quick and easy for the things I want to do.

               

              Cheers,
              Gary.

              Bernd Machenschalk
              Bernd Machenschalk
              Moderator
              Administrator
              Joined: 15 Oct 04
              Posts: 4312
              Credit: 250644838
              RAC: 34393

              Christian Beer wrote:This is

              27 Jul 2017 14:07:11 UTC
              Message 160346 in response to message 159507
              (moderation:
                )

                Christian Beer wrote:
                This is a BOINC wide problem. There is currently no way for a project to tell the Client that applications can be deleted. Other than manually fiddling with client_state

                In fact there is no way for the server to know which application files the client has. Instead IMHO the client would need to be modified to fetch the list of application (Web RPC applications page), then remove the application versions from its internal application list (that client_state.xml is an external representation of) that don't appear there and all application files that are not used in current (read: appearing on the applications page) applications the client already downloaded.

                BM

                AgentB
                AgentB
                Joined: 17 Mar 12
                Posts: 915
                Credit: 513211304
                RAC: 0

                Bernd Machenschalk wrote:In

                3 Aug 2017 20:35:41 UTC
                Message 160611 in response to message 160346
                (moderation:
                  )

                  Bernd Machenschalk wrote:
                  In fact there is no way for the server to know which application files the client has. Instead IMHO the client would need to be modified to fetch the list of application (Web RPC applications page), then remove the application versions from its internal application list (that client_state.xml is an external representation of) that don't appear there and all application files that are not used in current (read: appearing on the applications page) applications the client already downloaded.

                  I was thinking the client might be modified to keep track of when application/file was last used, and then remove it when asked to "cleanup old stuff".  This would give the client flexibility, as project admins might want to keep the apps available for reruns well into the future, but clients might need to recover the space.

                  I did toy with the idea of using the file system dates as being representative of an app being used, but the a full system backup kicked that idea into the long grass.

                  Comment viewing options

                  Select your preferred way to display the comments and click "Save settings" to activate your changes.

                  Forums › Problems and Bug Reports