Einstein disk usage growing

Bill
Bill
Joined: 2 Jun 17
Posts: 38
Credit: 281,753,699
RAC: 489,215
Topic 222551

I haven't tracked this problem closely, but I suspect that Einstein is using more and more disk space.  One of my systems it is using 2 GB, the other 4 GB.  I have total disk usage available to Boinc set high (the disk isn't being used for anything but crunching), and the work storage is set low (0.5 + 0.5).  When I am (or was) running MW & Seti, the amount of disk space these projects took up would seem to be fairly consistent.

I'm not sure if my problem is the same as mentioned here, but I suspect that Einstein has old tasks that have been crunched, but not deleted from my computer.  This isn't hurting anything, but I assumed that files would be deleted automatically.  I would think the project would keep its disk usage cleaned up; its not like the novice user would know that they have to reset the project, or worse, guess at what files need to be deleted.

Has anyone else noticed this?

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4,834
Credit: 17,999,433,478
RAC: 3,572,959

I think the scheduler needs

I think the scheduler needs to do some housekeeping on my project folder too.  Currently, E@h is taking up 33GB of disk space.

Pretty sure some of the datasets from last month aren't needed anymore since I haven't seen any work for those series lately.

Wish there was some way of forcing a cleanup other than a project reset.

 

mikey
mikey
Joined: 22 Jan 05
Posts: 12,295
Credit: 1,838,178,424
RAC: 4,666

They need to do the same

They need to do the same thing on their end as well....I STILL have workunits listed that were completed and validated on 4 June 2019!!!

 

h1_0145.35_O1C02Cl3In1__O1OD1I1_145.45Hz_145_0 406862063 12595872 4 Jun 2019 5:29:02 UTC 4 Jun 2019 20:56:48 UTC Completed and validated 13,601 13,126 1,000 Gravitational Wave Injection run on LIGO O1 Open Data v1.00 () windows_x86_64
h1_0145.85_O1C02Cl3In1__O1OD1I1_145.95Hz_145_1 406862073 12595872 4 Jun 2019 5:30:05 UTC 4 Jun 2019 21:36:16 UTC Completed and validated 15,306 13,403 1,000 Gravitational Wave Injection run on LIGO O1 Open Data v1.00 () windows_x86_64
h1_0144.75_O1C02Cl3In1__O1OD1I1_144.85Hz_136_4 406862051 12595872 4 Jun 2019 15:35:03 UTC 5 Jun 2019 18:48:52 UTC Completed and validated 42,391 18,934 1,000 Gravitational Wave Injection run on LIGO O1 Open Data v1.00 () windows_x86_64

 

I have not run those kinds of tasks in months and months.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,869
Credit: 113,201,152,120
RAC: 36,796,422

Bill wrote:...I'm not sure if

Bill wrote:
...I'm not sure if my problem is the same as mentioned here...

It's not a problem - it's just the way locality scheduling is supposed to work.  Yes, that thread is very relevant.  I spent a lot of time trying to explain why it's just the nature of the beast.  Please go back and read what I wrote there and please ask further questions if there is anything that needs further explanation.  In short, if you can afford the disk space, your best option is to let it be.

bill wrote:
... but I suspect that Einstein has old tasks ...

Absolutely NO!  There are no "old tasks" stored on your computer.  The space is occupied by data - BIG data!  You may very well need that data for more tasks in the future.  When the project deems that the data is really and truly finished with, the scheduler will issue a delete request.  You can use a project reset to get rid of everything but you will quickly get it all back again (in the form of huge bunches of different data files) so what's the point?  All you will effectively achieve is a waste of your own and the project's bandwidth.

bill wrote:
Has anyone else noticed this?

Everybody running any of the GW searches will notice it if they look.  The LIGO data is really, really big.  You are looking for a very small needle in a very big haystack.  Locality scheduling is an effort to limit the impact on your computer as you process as many tasks as possible with the least data possible.

Cheers,
Gary.

archae86
archae86
Joined: 6 Dec 05
Posts: 3,152
Credit: 7,131,294,931
RAC: 554,471

I suspect that this problem

I suspect that this problem is only a real problem for a pretty small fraction of users.  Gigabytes of storage are truly cheap these days and many of the participants likely have plenty to spare.

But for someone who truly does find this amount of storage "donated" to the project excessive, the alternative is not to exit Einstein, but to avoid the project applications that behave strongly this way.

The machine on which I am typing currently runs two "Navi 10" Radeon RX 5700 GPU cards to run Einstein Gamma-ray Pulsar binary search #1 tasks.  It has been running those tasks as the only BOINC work for over a year, though with some changes in hardware.

As of today, the entire c:\ProgramData\BOINC directory tree has just 1.1 Gigabytes.  That is less than 15 cents in US currency in value at the price I paid for a new boot SSD for another system recently.  Hard drives are even cheaper.

It is also far less storage than the reports in this thread arising from GW work.

I'm not advocating a flight from GW to GRP tasks for this reason among the general user population, just mentioning a way to stay with Einstein for people troubled by this matter.

 

Bill
Bill
Joined: 2 Jun 17
Posts: 38
Credit: 281,753,699
RAC: 489,215

Okay, if it is the way it is

Okay, if it is the way it is supposed to run, then I'm fine with it.  I was concerned that the disk usage was "running away".  If it kept growing and growing, some users I'm sure would eventually reach the maximum disk space they assign to Boinc, possibly causing other projects to not download tasks.  But, that doesn't seem to be the case (at least, not due to a bug), so I'll drop it.  I've got plenty of space, so that is not an issue either.

 

Lately I have not had the time to read through the Boinc forums as thoroughly as I have had in the past, so I appreciate the quick and brief response.  Thanks for taking the time to explain it, Gary and all! 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4,834
Credit: 17,999,433,478
RAC: 3,572,959

My 33GB of storage space is

My 33GB of storage space is NOT a small percentage of my disk space.  My partition is only 128GB in size on my SSD.

GW tasks are quickly forcing me into a decision to upgrade the storage from my 256GB SSD to a 512GB SSD.

Or I have to resize the two partitions on the drive and steal some space from my development partition.

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4,834
Credit: 17,999,433,478
RAC: 3,572,959

So Gary, why does my computer

So Gary, why does my computer need so many data files?  It is not as if I have a huge cache of work.  I limit the amount of work to only 25 cpu tasks and only 125 gpu tasks at any time.

 

San-Fernando-Valley
San-Fernando-Valley
Joined: 16 Mar 16
Posts: 290
Credit: 8,413,426,797
RAC: 7,018,205

Keith Myers wrote: My 33GB

Keith Myers wrote:

My 33GB of storage space is NOT a small percentage of my disk space.  My partition is only 128GB in size on my SSD.

GW tasks are quickly forcing me into a decision to upgrade the storage from my 256GB SSD to a 512GB SSD.

Or I have to resize the two partitions on the drive and steal some space from my development partition.

 

That is why I installed BOINC on a separate internal disk just for crunching.

This WD 2TB HDD disk cost me around 80 US dollars.

Enough performance for E@H.

BTW, LHC@home needs even more disk space ...

San-Fernando-Valley
San-Fernando-Valley
Joined: 16 Mar 16
Posts: 290
Credit: 8,413,426,797
RAC: 7,018,205

Keith Myers wrote: So Gary,

Keith Myers wrote:

So Gary, why does my computer need so many data files?  It is not as if I have a huge cache of work.  I limit the amount of work to only 25 cpu tasks and only 125 gpu tasks at any time.

I am wondering:  Hasn't that been explained ?

 

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4,834
Credit: 17,999,433,478
RAC: 3,572,959

San-Fernando-Valley

San-Fernando-Valley wrote:

Keith Myers wrote:

So Gary, why does my computer need so many data files?  It is not as if I have a huge cache of work.  I limit the amount of work to only 25 cpu tasks and only 125 gpu tasks at any time.

I am wondering:  Hasn't that been explained ?

 

No, it hasn't been explained.  Why is my computer an outlier?  Bill uses only uses 1.1GB for his 101 tasks.

Why does my computer need 33X more space for 45 more tasks.

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.