Gravitational wave search has filled my disk (I think)
9 Oct 2020 8:32:05 UTC
Topic 223680
(moderation:
I added a new computer at the beginning of September. Since then it has been running mostly Gravitational Wave searches. I noticed yesterday that it had swapped to Gamma-ray searches, whilst two other computers continued with Gravitational searches. This puzzled me so I took a look to see if I could work out why. It seems to be because the disk space I allowed is very nearly full. The Gravitational wave searches seem to have left behind files with names of the following two patterns:
-rw-r--r-- 1 boinc boinc 4032144 Sep 1 17:47 h1_0090.00_O2C02Cl5In0.m8Eh
-rw-r--r-- 1 boinc boinc 4096864 Sep 1 17:47 l1_0090.00_O2C02Cl5In0.m8Eh
There are 259 of each covering the dates from 1st Sep to October, and at 4MB each, very nearly fill the 2GB I had allowed. I looked at my other computers. They too are retaining these files, but are not running so many tasks and have more space available. It think it unlikely that these files are intended to fill up the disk until no more tasks can run, and so I should report a possible problem here?
Is there a problem?
Chris Littler wrote:Is there
)
I don't know how you decided to allow specifically 2 GB but at modern prices that is a tiny amount of disk. With the considerable amount of money you are probably paying for the electric power to run Einstein searches, surely you can afford to allocate a few pennies more worth of hard disk?
As to the other questions you raise:
1. I think it unlikely that some cleverness was involved in sending you, GR rather than gravity wave tasks just to save disk.
2. Where feasible Einstein avoids wasting network bandwidth sending you the same file twice by storing files on your system that have already been sent until it is clear they are no longer needed. Then they are deleted.
Unless you are running systems that have a critical shortage of disk capacity, I strongly suggest that you simply raise the limit. If you do have such a shortage, I suggest you restrict your Einstein contribution to Gamma-ray work by excluding Gravity Wave tasks.
[edited to add this: I just checked the BOINC directory in the machine on which I have been running 100% GW work for the last couple of months. It currently has 1351 files totalling 4 GB. Of that, 795 files totalling somewhat over 3 GB are 4 Megabyte files of the sort you mention. I continue to think your situation is normal apart from your unusual decision to impose a very tight disk usage limit]
Thank you for taking the time
)
Thank you for taking the time to assist.
How I came to allocate so little space to /var (on linux) is probably not important, but I wondered if my unusually low allocation is highlighting a problem early, that other people might hit later. But perhaps most people don't partition their disks and boinc using a few 10s of GB is no big deal. 2GB after a month is only ~25Gb after a year if it continues the same and perhaps this Gravitational wave search will be done by then and all tidied up.
I can and will swap some space around and give it lots more to use.
Actually I think some cleverness was involved in switching to Gamma ray as I can see the following in the Host Scheduler Log
This following is not not a
)
This following is not not a fix and is not recommended as a solution at all, but clicking a 'reset project' for Einstein would initially clear much of those files. Some of that load would definitely come back, but not necessarily all of that. I think it would depend on which part of the search area the project server would then send you new tasks after the reset. If you're lucky there had been already some files that were not really needed and maybe would've been deleted in near future anyway.
I've clicked a project reset occasionally (not very often) and sometimes it has sort of freed up space. The disk usage by Einstein has not started to climb up fast every time. I think I did a reset last week on my host, it's been then crunching GW GPU tasks and currently it still has only 340 MB in use by Einstein. Just to say... maybe for curiosity you could try a "one time" project reset and see how the fetus would then progress.
... just to pitch in my 2
)
... just to pitch in my 2 cents worth of experience:
I don't see any space allocs on my 8 rigs for E@H of more than 1GB (yes - one) on each when I check the DISK button in BOINC-Manager running GW or Gamma-ray WUs.
Maybe I'm missing out on something ...
Happy crunching!
I have over 3100 files of the
)
I have over 3100 files of the type Chris describes. According to BOINC manager the Einstein project is using 13.12 GB of space. I've allocated 15 GB to the project, but every once in a while (maybe again soon!) I raise the allocation if it looks like E@h wants more. As I understand it, E@h downloads many data sets once and then many tasks that process the data with various parameter choices. And as ARCHAE86 has pointed out, E@h will "eventually" delete those file. The oldest files, of this type, that I have are from August 9.
The GW CPU tasks of the "Spotlight" family typically use 350 to 850 MB of RAM; recently I am seeing tasks of a "CasA" family (CPU) that are using 1.1 to 1.9 GB of RAM. And looking at the "slots" information, for a "Spotlight" task there are soft links to 12 of those h1/l1 files; and for a "CasA" task there are 30 links. (Just for completness, a GPU GW task currently running a "Spotlight" task has 24 such soft links.)
The Ryzen hardware has a 256 GB SSD mass storage, with 38% currently in use, so giving E@h whatever it can use seems like the "right" thing to do.
Thank you all for the
)
Thank you all for the comparisons. I shall stop worrying and allocate some extra disk space :-)