Excessive Disk Usage

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2143
Credit: 2952336848
RAC: 699370

RE: The easiest way to get

Message 97310 in response to message 97308

Quote:

The easiest way to get rid of the files of course if you don't care about bandwidth is just to reset the project (update before to make sure completed tasks are reported). You will be resent the tasks that your client got assigned before and download the files for these tasks again.

BM


Buried somewhere deep in either this or a parallel thread is my report to Jord, having just discovered that a project reset doesn't remove the file references from client_state.xml - so not only do you get re-assigned the tasks and the files that they need, you also re-download (at the next restart) all the data files you had before.

David, don't feel any need to apologise. Sometimes it happens that a simple question prompts a fresh appraisal of something that everybody thought they knew already - and it turns out not to be as simple as it seemed. It's that fresh appraisal which has taught me things I didn't know before, led to the dvelopment of workrounds for people who really do need their space back urgently, and corrected a bug in the project handling of these data files. Give yourself a pat on the back!

PS It was this thread. And David's question about eight posts in should have given the game away much sooner:

Quote:
It's now up to 237MB after reseting the project.... Is that normal ???
Jord
Joined: 26 Jan 05
Posts: 2952
Credit: 5893653
RAC: 167

RE: RE: The easiest way

Message 97311 in response to message 97310

Quote:
Quote:

The easiest way to get rid of the files of course if you don't care about bandwidth is just to reset the project (update before to make sure completed tasks are reported). You will be resent the tasks that your client got assigned before and download the files for these tasks again.

BM


Buried somewhere deep in either this or a parallel thread is my report to Jord, having just discovered that a project reset doesn't remove the file references from client_state.xml - so not only do you get re-assigned the tasks and the files that they need, you also re-download (at the next restart) all the data files you had before.


A "simple" workaround for that is to delete all entries in client_state.xml, but for one. Preferably the one you're running already. Then you get only that data-set re-sent.

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2143
Credit: 2952336848
RAC: 699370

RE: RE: RE: The easiest

Message 97312 in response to message 97311

Quote:
Quote:
Quote:

The easiest way to get rid of the files of course if you don't care about bandwidth is just to reset the project (update before to make sure completed tasks are reported). You will be resent the tasks that your client got assigned before and download the files for these tasks again.

BM


Buried somewhere deep in either this or a parallel thread is my report to Jord, having just discovered that a project reset doesn't remove the file references from client_state.xml - so not only do you get re-assigned the tasks and the files that they need, you also re-download (at the next restart) all the data files you had before.

A "simple" workaround for that is to delete all entries in client_state.xml, but for one. Preferably the one you're running already. Then you get only that data-set re-sent.


Jord, no. A - the minimum quantum of task allocation - is not the same as a . The "one you're running already", if it's S5GCE, will reference at least 28 different files via . That's what makes it difficult (and why BOINC tries to hang on to those files for dear life, and avoid re-downloading them if at all possible: 120 MB or so is a stress, not only for the home user, but for the project as well).

Jord
Joined: 26 Jan 05
Posts: 2952
Credit: 5893653
RAC: 167

That's why BM says that not

Message 97313 in response to message 97312

That's why BM says that not everyone should try this, but only those who know (just about) what they're doing. And even then, probably not, as it'll clear itself up by now, giving it enough patience.

David
David
Joined: 18 Feb 10
Posts: 9
Credit: 50931
RAC: 0

RE: RE: If you wanted

Message 97314 in response to message 97287

Quote:
Quote:

If you wanted to experiment and didn't mind BOINC telling you from time to time that it couldn't get more work because of lack of disk space, you could try carefully reducing your disk space preferences until you got that message. You could then observe if your BOINC client was able to delete enough of your excess S5R7 files to meet the disk space restrictions you have now imposed.

Um,

My preferences state "Use at most 0.05 GB disk space" which is equivalent to 50 odd MB..... Current Disk usage is 334.72MB..... It's no big problem, but, I find it odd.

Gundolf Jahn
Gundolf Jahn
Joined: 1 Mar 05
Posts: 1079
Credit: 341280
RAC: 0

RE: My preferences state

Message 97315 in response to message 97314

Quote:
My preferences state "Use at most 0.05 GB disk space" which is equivalent to 50 odd MB..... Current Disk usage is 334.72MB..... It's no big problem, but, I find it odd.


Did you check your local preferences? They override the global ones.

What does BOINC report at startup? Mine says:04-Apr-2010 12:10:09 [---] Preferences limit disk usage to 1.86GB
Gruß,
Gundolf

Computer sind nicht alles im Leben. (Kleiner Scherz)

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2143
Credit: 2952336848
RAC: 699370

Just to confirm - the new

Just to confirm - the new deletion code is working:

Einstein@Home	14/04/2010 18:20:56	Got server request to delete file h1_0457.75_S5R4
Einstein@Home	14/04/2010 18:20:56	Got server request to delete file l1_0457.75_S5R4
Einstein@Home	14/04/2010 18:20:56	Got server request to delete file h1_0457.75_S5R7
Einstein@Home	14/04/2010 18:20:56	Got server request to delete file l1_0457.75_S5R7


So: if you can live with your current disk usage, it shouldn't - in the long term - get any worse.

David
David
Joined: 18 Feb 10
Posts: 9
Credit: 50931
RAC: 0

RE: RE: My preferences

Message 97317 in response to message 97315

Quote:
Quote:
My preferences state "Use at most 0.05 GB disk space" which is equivalent to 50 odd MB..... Current Disk usage is 334.72MB..... It's no big problem, but, I find it odd.

Did you check your local preferences? They override the global ones.

What does BOINC report at startup? Mine says:04-Apr-2010 12:10:09 [---] Preferences limit disk usage to 1.86GB
Gruß,
Gundolf

Ah...
Mine says 15/04/2010 Preferences limit disk usage to 50.00GB

Gundolf Jahn
Gundolf Jahn
Joined: 1 Mar 05
Posts: 1079
Credit: 341280
RAC: 0

RE: Ah... Mine says

Message 97318 in response to message 97317

Quote:
Ah...
Mine says 15/04/2010 Preferences limit disk usage to 50.00GB


So much to "My preferences state 'Use at most 0.05 GB disk space' which is equivalent to 50 odd MB..." ;-)

Gruß,
Gundolf

Computer sind nicht alles im Leben. (Kleiner Scherz)

David
David
Joined: 18 Feb 10
Posts: 9
Credit: 50931
RAC: 0

RE: RE: From my first

Message 97319 in response to message 97304

Quote:
Quote:
From my first list, I can safely lose all the _0295, _0416.65 to _0416.75, and that pesky _1071.25 off the bottom. OK so far, Bernd?

The trouble is only with _S5R7 files. As long as you have a matching _S5R4 file on your machine it will be taken care of by the new scheduler, i.e. it might be needed by tasks you have or might get and will be automatically deleted when the corresponding _S5R4 file gets deleted. h1_1071.25_S5R4 (_S5R4 file above 1000.00) is from S5R6, it should eventually get deleted automatically, too. Obsolete files that could be deleted manually are _S5R7 files of [bold]0295[/bold] and the [bold]_0416[/bold] ones that don't have a corresponding _S5R4 file.

BM


Hi,
I've noticed that my disk usage is now up to 480MB. It appears that I have orphaned S5R7 files. Though the files that are orphaned for me are ".0330." & ".0380." files (as opposed to ".0295." or ".0416." as mentioned above by Bernd). Looking at my client_state.xml (and the directory listing), I can see that I also have a lone "l1_0264.40_S5R7" file (without a corresponding "h1_0264.40_S5R7" file ??). I have not yet manually deleted anything, though I think I understand what I need to do to reclaim the space taken by the orphaned S5R7 files (I can't see any orphaned S5R4 files...).

1: Completely stop BOINC.
2: Backup "{BOINC projects}\einstein.phys.uwm.edu" & "client_state.xml"
3: Physically delete orphaned S5R7 files from {BOINC projects}\einstein.phys.uwm.edu.
4: Edit client_state.xml and remove the [file_info] blocks for the orphaned S5R7 files.

Thanks for everyone's help...
David

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.