S4 search will take more than 200 days to complete

pisi78
pisi78
Joined: 22 Jan 05
Posts: 4
Credit: 380,361
RAC: 0

RE: Hmmm, yes ... But the

Message 24631 in response to message 24630

Quote:

Hmmm, yes ...

But the crisis has passed. The last time I looked, yep, 197.1 days ...

So, we will be done way before 200 days ... :)

I have been altering my set-up though to put more here ... should be shifting to nearly 56.07% to Einstein@Home ...

At this rate, we crunch roughly the 0,5% in a day. so my estimate is 166 days :)

ExtraTerrestrial Apes
ExtraTerrestria...
Joined: 10 Nov 04
Posts: 770
Credit: 551,957,235
RAC: 202,015

I was just about to suggest

I was just about to suggest using an initial replication of 3 for Einstein, since with a 200 days search you can easily live with higher result latency if you get a higher throughput for it. Last time I checked this on my machine it was at 4. Well, it's already at 3 now, so no complaining from me here ;)

MrS

Scanning for our furry friends since Jan 2002

RandyC
RandyC
Joined: 18 Jan 05
Posts: 6,238
Credit: 111,139,797
RAC: 0

RE: I was just about to

Message 24633 in response to message 24632

Quote:

I was just about to suggest using an initial replication of 3 for Einstein, since with a 200 days search you can easily live with higher result latency if you get a higher throughput for it. Last time I checked this on my machine it was at 4. Well, it's already at 3 now, so no complaining from me here ;)

MrS

When they started using the new Albert WUs, they lowered the initial replication count.

Seti Classic Final Total: 11446 WU.

Ben Owen
Ben Owen
Joined: 21 Dec 04
Posts: 117
Credit: 65,961,833
RAC: 1,067

Paul Buck wrote: RE: Ok,

Paul Buck wrote:

Quote:

Ok, now the question is, why does this matter?

Other than you would like to get it done sooner. :)

WHat I mean is, is S5 running now, and by 200 days from now we will be collecting data for S6 even though we have not started processing the data for S5?

That is the only point that is not clear. All of the major projects would like more people. Heck, as far as I know the only project that may be at saturation (for the moment) is SETI@Home though I stopped doing work there so this may no longer be an issue ... :)

I normally hang out on the Science board, but this is an important question.

For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.

This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.

But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.

And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.

Thank you very much,
Ben

Jayargh
Jayargh
Joined: 9 Feb 05
Posts: 64
Credit: 1,205,159
RAC: 0

RE: I normally hang out on

Message 24635 in response to message 24634


Quote:


I normally hang out on the Science board, but this is an important question.

For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.

This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.

But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.

And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.

Thank you very much,
Ben

Ben I think we can make up the other 7/8s if you get the gist of my post Einstein Low Credit

pisi78
pisi78
Joined: 22 Jan 05
Posts: 4
Credit: 380,361
RAC: 0

RE: Paul Buck

Message 24636 in response to message 24634

Quote:
Paul Buck wrote:
Quote:

Ok, now the question is, why does this matter?

Other than you would like to get it done sooner. :)

WHat I mean is, is S5 running now, and by 200 days from now we will be collecting data for S6 even though we have not started processing the data for S5?

That is the only point that is not clear. All of the major projects would like more people. Heck, as far as I know the only project that may be at saturation (for the moment) is SETI@Home though I stopped doing work there so this may no longer be an issue ... :)

I normally hang out on the Science board, but this is an important question.

For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.

This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.

But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.

And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.

Thank you very much,
Ben

thanks, it's important for a DC project to keep the volunteers informed :)

I hope that the number of computers will increase, but another important step is to optimize the application even further.
Every gain in application performance is available immediately within all hosts.

The problems with this kind of project is that there are many things to do and the human resources are limited.

You could ask for the help of volunteer programmers, within the 50.000 active users there will be surely some very skilled programmers to do hand coding like akosf did.

The code shouldn't be opensource to avoid problems, but every volunteer could sign a NDA.

gravitysmith
gravitysmith
Joined: 8 Nov 04
Posts: 55
Credit: 91,013,391
RAC: 8,195

RE: For one thing, if it

Message 24637 in response to message 24634

Quote:

For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.

Ben, thanks for your response!

I have an add-on question though. What is the rush? Presumably the data is stored somewhere and can be analyzed at any time. Therefore, given enough time, all the data can be analyzed to the extent that computations are no longer the limit.

Is it the case that the next data run somehow depends upon the results of the previous run? Are the current results being used to guide the improvements for the next iteration of the experiment?

Thanks.
smith

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9,352,143
RAC: 0

RE: I have an add-on

Message 24638 in response to message 24637

Quote:


I have an add-on question though. What is the rush? Presumably the data is stored somewhere and can be analyzed at any time. Therefore, given enough time, all the data can be analyzed to the extent that computations are no longer the limit.

Is it the case that the next data run somehow depends upon the results of the previous run? Are the current results being used to guide the improvements for the next iteration of the experiment?

Thanks.
smith

One reason which comes to mind is when you go to the powers-that-be hat in hand looking for funding, they usually like to see pretty pictoral representations and humungous technical volumes showing the current results of your research.

:-)

As the saying goes, "No Bucks...., No Buck Rogers!"

Alinator

AMD-USR_JL
AMD-USR_JL
Joined: 24 Jan 06
Posts: 13
Credit: 27,886
RAC: 0

RE: I normally hang out on

Message 24639 in response to message 24634

Quote:


I normally hang out on the Science board, but this is an important question.

For one thing, if it takes us 200 days to analyze every 600 hours of data, we are running a factor of 8 too slow to keep up with the data. Or you can flip it and say we can only analyze 1/8 of the data.

This directly affects our science goal. You could view it as reducing the range to which we could see a nearby neutron star which is invisible but emitting gravitational waves. It's hard to quantify in an easily explainable way since a lot of factors come in, like how much data we analyze, whether 600 hours comes in 60x10-hour chunks or 20x30-hour chunks (the latter is better), what frequency band and frequency changes we can look at, what age object we can look at. For that matter, we haven't even properly quantified it at the technical journal article level, though we're working on it.

But it is a qualitatively true statement that we are computationally limited. There is a limit on the range of the search set by the operation of the instrument - its noise spectrum, duty cycle, noise statistics - which would exist even if we had all the computing power in the world, and we are nowhere near that limit. But every CPU donating cycles gets us closer.

And it is also a true statement that we very much appreciate every CPU cycle you can spare. When I was a grad student, the all-sky pulsar search was considered absolutely hopeless. Now we are actually doing it, at some non-ridiculous fraction of the instrumental limit, and it would not be happening without you.

Thank you very much,
Ben


According to Moore's Law we should double our crunching rate every 18 months. So in 18 months we will only be taking 100 days to crunch 600 hours. In another 18 it will 50 days. Finally in another 18 it will reach 25, which is our target of 600 hours.It will take 54 months or 4 years and 6 months to reach 25 days.
But like you said einstein has not reached everyone in the world. There's only 176,459 users in the database. Considering there was 6 billion people in the world in 2000, then eistein has hardly reached anyone. So if the number of crunchers and crunching rate doubled every 18 months, then it would only take 36 months before einstein can crunch the data as fast as they collect it.
This would happen because if the crunchers doubled then it would be 100 days. Those crunchers' rates would double and make it 50.
I think that we just need more advertising, because the number of users doubling could happen a lot quicker than 18 months. It could even triple or even quadruple.

Jordan Wilberding
Jordan Wilberding
Joined: 19 Feb 05
Posts: 162
Credit: 715,454
RAC: 0

RE: According to

Message 24640 in response to message 24639

Quote:

According to Moore's Law we should double our crunching rate every 18 months.

I am not sure Moore's Law applies to crunching rates :)

such things just should not be writ so please destroy this if you wish to live 'tis better in ignorance to dwell than to go screaming into the abyss worse than hell

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.