Bruce,
Perhaps 7 days is too long to give someone to examine their completed units and credit. I suspect that those who are interested will look every day, and those who don't, couldn't care less.
A few months ago Seti started deleting after a few minutes, which was clearly unacceptable, and then relented. Now they leave the units on for months and are clogged up.
I suggest you reduce the period bit by bit until people notice and complain. But then there is the problem of holidays...
I presume these records are available for download by the external collectors of such data and thus could be viewable on servers outside of the project?
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
I suspect that seven days is a resonable value. If you consider slow machines, multiple projects and as Rubernicus remarked "But then there is the problem of holidays...", you need some elbow room. BUT, don't make it any larger though. My "pending" queue is large enough thank you...
I am one of those people who very rarely looks at his complete WU so it would not bother me a bit to purge the completed WU sooner than 7 days. On the other hand, I don't have any trouble with getting WU's returned within the alloted time limit, so lengthening the report deadline is a moot issue for me too.
sorry, that probably didn't help much.
Jim
> Bruce,
> Perhaps 7 days is too long to give someone to examine their completed units
> and credit. I suspect that those who are interested will look every day, and
> those who don't, couldn't care less.
> A few months ago Seti started deleting after a few minutes, which was clearly
> unacceptable, and then relented. Now they leave the units on for months and
> are clogged up.
> I suggest you reduce the period bit by bit until people notice and complain.
> But then there is the problem of holidays...
>
As i stated earlier i am all for shortening the time completed WU are stored in the database, if the resources freed by this can be used for something more useful for the project. Maybe leaving them on the server for 24-48H will be sufficient for the users that’s interested and for the stat sites to collect the information. One thing thou, how will this effect WU’s that are returned late? If i understand thing correctly, these are still useful for the project.
Then you're really interested in a subject, there is no way to avoid it. You have to read the Manual.
> As i stated earlier i am all for shortening the time completed WU are stored
> in the database, if the resources freed by this can be used for something more
> useful for the project. Maybe leaving them on the server for 24-48H will be
> sufficient for the users that’s interested and for the stat sites to collect
> the information. One thing thou, how will this effect WU’s that are returned
> late? If i understand thing correctly, these are still useful for the project.
>
If they are returned before the replacement is sent, then clearly they are useful to the project. If they are returned after this, but before the replacement is returned they have some value as insurance against the replacement not being returned or being bad. After the WU has been validated, there is probably very little value left for the late WU.
I thought I read earlier that the reason for keeping the deadline to 1 week was to ensure that the WU's were returned quickly to aid in debugging, and that when the project went public, the deadline would be relaxed. No one said anything about keeping the deadline short because the database would get too large. Which is it?
Edit-- OK, I found the reference way back in the beginning of the thread. Never mind...
After reading the comments, I'm still undecided about what to do. So in the short term I'm not going to do anything. In the longer term, the option of having a shorter time before completed WU are purged from the database, and a longer deadline, seems reasonable. I'll mull this over.
[Oh, in response to one question: no, after the WU is purged from the database, it is only available in our archives, not to other projects, statistics servers, etc.]
> This is indeed a very clear analysis of the situation.
> >So what could be done to reduce the size of the database? If i am correct, > >the database that we are talking about hear are the database that consists
> >of entries of what host have downloaded what WU. Then you download a WU an
> >entry is made in the database. This entry remains in the database until all
> >other users who have downloaded the same WU have bean accounted for or the
> >deadline for unaccounted users have bean reached. My question is, are the > >entries in the database removed a.s.a.p. or is the removal delayed a couple > >of days so we can look at some past results? If so, i am willing to trade
> >that feature for a couple of hour’s extension to the deadline.
>
> I'd appreciate feedback on this. I decided to leave completed WU in the
> database for 7 days, to let our participants go back and examine them.
> Reducing this storage time would indeed help to keep the database 'lean and
> mean'. Would less than 7 days be OK? What do you propose?
>
> Cheers,
> Bruce
Bruce, et al,
I'd cut it down to 3-4 days to look at completed results and increase the WU deadline by 3 days. Unless you begin adding some graphics to this feature a'la CPDN, I don't find it particularly useful.
I tend to work on the minimum amount of time then double it. I believe 2 days is sufficient to view returned wu's so double it to 4 days. But I thought during the hype and fresh enthusiasm that always greets BOINC projects, you were talking about getting X number of users (I forget what it was) and that you were going to put the deadline out to 2 weeks. This really sets the maximum database size if you were going to allow viewing of completed wu's for 7 days. Has anything changed? Have you come to grips with how DB intensive BOINC really is and how you are limited by this?
Paul.
The Final Front Ear.
ps. If this sounds too direct it probably is. The darn cat woke me at 2am playing with a mouse out the front of the bedroom door after only getting 3 hrs sleep the night before. And I find myself at 2:25am writing this hoping to get sleepy again, so my levels of tact and forthrightedness (is that a word) are totally fubar'd...but hopefully you get my drift.
Bruce, Perhaps 7 days is too
)
Bruce,
Perhaps 7 days is too long to give someone to examine their completed units and credit. I suspect that those who are interested will look every day, and those who don't, couldn't care less.
A few months ago Seti started deleting after a few minutes, which was clearly unacceptable, and then relented. Now they leave the units on for months and are clogged up.
I suggest you reduce the period bit by bit until people notice and complain. But then there is the problem of holidays...
I presume these records are
)
I presume these records are available for download by the external collectors of such data and thus could be viewable on servers outside of the project?
Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.
I suspect that seven days is
)
I suspect that seven days is a resonable value. If you consider slow machines, multiple projects and as Rubernicus remarked "But then there is the problem of holidays...", you need some elbow room. BUT, don't make it any larger though. My "pending" queue is large enough thank you...
Ned
Ol' Retired IT Geezer
I am one of those people who
)
I am one of those people who very rarely looks at his complete WU so it would not bother me a bit to purge the completed WU sooner than 7 days. On the other hand, I don't have any trouble with getting WU's returned within the alloted time limit, so lengthening the report deadline is a moot issue for me too.
sorry, that probably didn't help much.
Jim
> Bruce,
> Perhaps 7 days is too long to give someone to examine their completed units
> and credit. I suspect that those who are interested will look every day, and
> those who don't, couldn't care less.
> A few months ago Seti started deleting after a few minutes, which was clearly
> unacceptable, and then relented. Now they leave the units on for months and
> are clogged up.
> I suggest you reduce the period bit by bit until people notice and complain.
> But then there is the problem of holidays...
>
Jim
As i stated earlier i am all
)
As i stated earlier i am all for shortening the time completed WU are stored in the database, if the resources freed by this can be used for something more useful for the project. Maybe leaving them on the server for 24-48H will be sufficient for the users that’s interested and for the stat sites to collect the information. One thing thou, how will this effect WU’s that are returned late? If i understand thing correctly, these are still useful for the project.
Then you're really interested in a subject, there is no way to avoid it. You have to read the Manual.
> As i stated earlier i am
)
> As i stated earlier i am all for shortening the time completed WU are stored
> in the database, if the resources freed by this can be used for something more
> useful for the project. Maybe leaving them on the server for 24-48H will be
> sufficient for the users that’s interested and for the stat sites to collect
> the information. One thing thou, how will this effect WU’s that are returned
> late? If i understand thing correctly, these are still useful for the project.
>
If they are returned before the replacement is sent, then clearly they are useful to the project. If they are returned after this, but before the replacement is returned they have some value as insurance against the replacement not being returned or being bad. After the WU has been validated, there is probably very little value left for the late WU.
BOINC WIKI
I thought I read earlier that
)
I thought I read earlier that the reason for keeping the deadline to 1 week was to ensure that the WU's were returned quickly to aid in debugging, and that when the project went public, the deadline would be relaxed. No one said anything about keeping the deadline short because the database would get too large. Which is it?
Edit-- OK, I found the reference way back in the beginning of the thread. Never mind...
After reading the comments,
)
After reading the comments, I'm still undecided about what to do. So in the short term I'm not going to do anything. In the longer term, the option of having a shorter time before completed WU are purged from the database, and a longer deadline, seems reasonable. I'll mull this over.
[Oh, in response to one question: no, after the WU is purged from the database, it is only available in our archives, not to other projects, statistics servers, etc.]
Cheers,
Bruce
Director, Einstein@Home
> This is indeed a very clear
)
> This is indeed a very clear analysis of the situation.
> >So what could be done to reduce the size of the database? If i am correct, > >the database that we are talking about hear are the database that consists
> >of entries of what host have downloaded what WU. Then you download a WU an
> >entry is made in the database. This entry remains in the database until all
> >other users who have downloaded the same WU have bean accounted for or the
> >deadline for unaccounted users have bean reached. My question is, are the > >entries in the database removed a.s.a.p. or is the removal delayed a couple > >of days so we can look at some past results? If so, i am willing to trade
> >that feature for a couple of hour’s extension to the deadline.
>
> I'd appreciate feedback on this. I decided to leave completed WU in the
> database for 7 days, to let our participants go back and examine them.
> Reducing this storage time would indeed help to keep the database 'lean and
> mean'. Would less than 7 days be OK? What do you propose?
>
> Cheers,
> Bruce
Bruce, et al,
I'd cut it down to 3-4 days to look at completed results and increase the WU deadline by 3 days. Unless you begin adding some graphics to this feature a'la CPDN, I don't find it particularly useful.
Bruce, I tend to work on
)
Bruce,
I tend to work on the minimum amount of time then double it. I believe 2 days is sufficient to view returned wu's so double it to 4 days. But I thought during the hype and fresh enthusiasm that always greets BOINC projects, you were talking about getting X number of users (I forget what it was) and that you were going to put the deadline out to 2 weeks. This really sets the maximum database size if you were going to allow viewing of completed wu's for 7 days. Has anything changed? Have you come to grips with how DB intensive BOINC really is and how you are limited by this?
Paul.
The Final Front Ear.
ps. If this sounds too direct it probably is. The darn cat woke me at 2am playing with a mouse out the front of the bedroom door after only getting 3 hrs sleep the night before. And I find myself at 2:25am writing this hoping to get sleepy again, so my levels of tact and forthrightedness (is that a word) are totally fubar'd...but hopefully you get my drift.
Paul
