I was just about to suggest using an initial replication of 3 for Einstein, since with a 200 days search you can easily live with higher result latency if you get a higher throughput for it. Last time I checked this on my machine it was at 4. Well, it's already at 3 now, so no complaining from me here ;)
I was just about to suggest using an initial replication of 3 for Einstein, since with a 200 days search you can easily live with higher result latency if you get a higher throughput for it. Last time I checked this on my machine it was at 4. Well, it's already at 3 now, so no complaining from me here ;)
MrS
When they started using the new Albert WUs, they lowered the initial replication count.
Other than you would like to get it done sooner. :)
WHat I mean is, is S5 running now, and by 200 days from now we will be collecting data for S6 even though we have not started processing the data for S5?
That is the only point that is not clear. All of the major projects would like more people. Heck, as far as I know the only project that may be at saturation (for the moment) is SETI@Home though I stopped doing work there so this may no longer be an issue ... :)
I normally hang out on the Science board, but this is an important question.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.
But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.
And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.
I normally hang out on the Science board, but this is an important question.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.
But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.
And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.
Thank you very much,
Ben
Ben I think we can make up the other 7/8s if you get the gist of my post Einstein Low Credit
Other than you would like to get it done sooner. :)
WHat I mean is, is S5 running now, and by 200 days from now we will be collecting data for S6 even though we have not started processing the data for S5?
That is the only point that is not clear. All of the major projects would like more people. Heck, as far as I know the only project that may be at saturation (for the moment) is SETI@Home though I stopped doing work there so this may no longer be an issue ... :)
I normally hang out on the Science board, but this is an important question.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.
But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.
And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.
Thank you very much,
Ben
thanks, it's important for a DC project to keep the volunteers informed :)
I hope that the number of computers will increase, but another important step is to optimize the application even further.
Every gain in application performance is available immediately within all hosts.
The problems with this kind of project is that there are many things to do and the human resources are limited.
You could ask for the help of volunteer programmers, within the 50.000 active users there will be surely some very skilled programmers to do hand coding like akosf did.
The code shouldn't be opensource to avoid problems, but every volunteer could sign a NDA.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
Ben, thanks for your response!
I have an add-on question though. What is the rush? Presumably the data is stored somewhere and can be analyzed at any time. Therefore, given enough time, all the data can be analyzed to the extent that computations are no longer the limit.
Is it the case that the next data run somehow depends upon the results of the previous run? Are the current results being used to guide the improvements for the next iteration of the experiment?
I have an add-on question though. What is the rush? Presumably the data is stored somewhere and can be analyzed at any time. Therefore, given enough time, all the data can be analyzed to the extent that computations are no longer the limit.
Is it the case that the next data run somehow depends upon the results of the previous run? Are the current results being used to guide the improvements for the next iteration of the experiment?
Thanks.
smith
One reason which comes to mind is when you go to the powers-that-be hat in hand looking for funding, they usually like to see pretty pictoral representations and humungous technical volumes showing the current results of your research.
:-)
As the saying goes, "No Bucks...., No Buck Rogers!"
I normally hang out on the Science board, but this is an important question.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.
But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.
And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.
Thank you very much,
Ben
According to Moore's Law we should double our crunching rate every 18 months. So in 18 months we will only be taking 100 days to crunch 600 hours. In another 18 it will 50 days. Finally in another 18 it will reach 25, which is our target of 600 hours.It will take 54 months or 4 years and 6 months to reach 25 days.
But like you said einstein has not reached everyone in the world. There's only 176,459 users in the database. Considering there was 6 billion people in the world in 2000, then eistein has hardly reached anyone. So if the number of crunchers and crunching rate doubled every 18 months, then it would only take 36 months before einstein can crunch the data as fast as they collect it.
This would happen because if the crunchers doubled then it would be 100 days. Those crunchers' rates would double and make it 50.
I think that we just need more advertising, because the number of users doubling could happen a lot quicker than 18 months. It could even triple or even quadruple.
According to Moore's Law we should double our crunching rate every 18 months.
I am not sure Moore's Law applies to crunching rates :)
such things just should not be writ so please destroy this if you wish to live 'tis better in ignorance to dwell than to go screaming into the abyss worse than hell
RE: Hmmm, yes ... But the
)
At this rate, we crunch roughly the 0,5% in a day. so my estimate is 166 days :)
I was just about to suggest
)
I was just about to suggest using an initial replication of 3 for Einstein, since with a 200 days search you can easily live with higher result latency if you get a higher throughput for it. Last time I checked this on my machine it was at 4. Well, it's already at 3 now, so no complaining from me here ;)
MrS
Scanning for our furry friends since Jan 2002
RE: I was just about to
)
When they started using the new Albert WUs, they lowered the initial replication count.
Seti Classic Final Total: 11446 WU.
Paul Buck wrote: RE: Ok,
)
Paul Buck wrote:
I normally hang out on the Science board, but this is an important question.
For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.
This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.
But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.
And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.
Thank you very much,
Ben
RE: I normally hang out on
)
Ben I think we can make up the other 7/8s if you get the gist of my post Einstein Low Credit
RE: Paul Buck
)
thanks, it's important for a DC project to keep the volunteers informed :)
I hope that the number of computers will increase, but another important step is to optimize the application even further.
Every gain in application performance is available immediately within all hosts.
The problems with this kind of project is that there are many things to do and the human resources are limited.
You could ask for the help of volunteer programmers, within the 50.000 active users there will be surely some very skilled programmers to do hand coding like akosf did.
The code shouldn't be opensource to avoid problems, but every volunteer could sign a NDA.
RE: For one thing, if it
)
Ben, thanks for your response!
I have an add-on question though. What is the rush? Presumably the data is stored somewhere and can be analyzed at any time. Therefore, given enough time, all the data can be analyzed to the extent that computations are no longer the limit.
Is it the case that the next data run somehow depends upon the results of the previous run? Are the current results being used to guide the improvements for the next iteration of the experiment?
Thanks.
smith
RE: I have an add-on
)
One reason which comes to mind is when you go to the powers-that-be hat in hand looking for funding, they usually like to see pretty pictoral representations and humungous technical volumes showing the current results of your research.
:-)
As the saying goes, "No Bucks...., No Buck Rogers!"
Alinator
RE: I normally hang out on
)
According to Moore's Law we should double our crunching rate every 18 months. So in 18 months we will only be taking 100 days to crunch 600 hours. In another 18 it will 50 days. Finally in another 18 it will reach 25, which is our target of 600 hours.It will take 54 months or 4 years and 6 months to reach 25 days.
But like you said einstein has not reached everyone in the world. There's only 176,459 users in the database. Considering there was 6 billion people in the world in 2000, then eistein has hardly reached anyone. So if the number of crunchers and crunching rate doubled every 18 months, then it would only take 36 months before einstein can crunch the data as fast as they collect it.
This would happen because if the crunchers doubled then it would be 100 days. Those crunchers' rates would double and make it 50.
I think that we just need more advertising, because the number of users doubling could happen a lot quicker than 18 months. It could even triple or even quadruple.
RE: According to
)
I am not sure Moore's Law applies to crunching rates :)
such things just should not be writ so please destroy this if you wish to live 'tis better in ignorance to dwell than to go screaming into the abyss worse than hell