Will there be a S6?

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,525
Credit: 281,477,356
RAC: 46,642

RE: After reading all this,

Message 58175 in response to message 58174

Quote:
After reading all this, I'm still not clear on the future. We're soon to complete S5, right?


S5 data collection at the detectors finishes ~ the middle of this calendar year. The first S5 data analysis, 'S5R1' if you like, finishes in ~ a month ( perhaps a bit longer given recent server troubles ).

There isn't a one-to-one relationship between data collection and analysis because of the many different things that could be searched for. We'll be working on new/enlarged data sets at E@H soon - the software details, data sets etc are being hammered into shape at present. One point made at the AEI presentation by Bruce Allen was to keep the crunching parameters ( file sizes etc ) suitable for the lower speed connections around the world. I gather broadband is NOT the majority mode of communication but dial-up IS.

Quote:
Are we soon to also run out of crunching work for E@H?

No, plenty of science yet.

Quote:
Will there be any crunching down-time for us?

No, servers willing.

Quote:
And if so, when and for how long?

Not-applicable.

Just imagine Dirk Hartog ( or insert your favourite sea-farer here ) returning with a huge list of depth soundings, times/dates, latitudes/longitudes from a round the world trip ..... it'll take a while to do the map and deduce the geography. No shortage of cartography work .... :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,833
Credit: 108,247,871,250
RAC: 32,777,105

RE: After reading all this,

Message 58176 in response to message 58174

Quote:
After reading all this, I'm still not clear on the future. We're soon to complete S5, right? Are we soon to also run out of crunching work for E@H? Will there be any crunching down-time for us? And if so, when and for how long?

We are soon to complete S5R1. I imagine S5R2 will follow. There will be no break if past performance is any sort of a guide.

The front page blurb says we are crunching the most sensitive 840 hours of data from S5. If we allow a factor of 0.75 for the time that science mode is achieved by the detectors, those 840 hours would have taken 1120 wall clock hours to accumulate - ie less than 7 weeks. They have been accumulating S5 data for over a year now (i believe). To me that seems to say that they are accumulating data much faster than we are crunching it. I'm sure somebody will correct me if I'm wrong but I think it's just about impossible to run out of data to crunch :).

Cheers,
Gary.

zombie67 [MM]
Joined: 10 Oct 06
Posts: 109
Credit: 451,368,753
RAC: 82

Thanks for the answers!

Thanks for the answers!

Reno, NV
Team: SETI.USA

XUL:
XUL:
Joined: 18 Oct 06
Posts: 5
Credit: 207,634
RAC: 0

RE: One point made at the

Message 58178 in response to message 58175

Quote:
One point made at the AEI presentation by Bruce Allen was to keep the crunching parameters ( file sizes etc ) suitable for the lower speed connections around the world. I gather broadband is NOT the majority mode of communication but dial-up IS.

hmmm. how about some kind of "poll", asking users what bandwidth they can assure? or, i could imagine - thou hast to code it! ;) - some preference/setting in the client "prefer BIG chunks" / "rather like it small and handy"

i mean, after some time it would be clear were to put focus on rather than optimising peripherial questions...
though, i do not know how deep this approach of reducing file size impacts on the whole math behind.

one thing doesn't sound logical too much: why reducing size? you'll have to get/return it anyway at some time, and the retrieving/sending of data can be made asynchronious (once an up-/download fails by connection loss, next time transfers would be resumed... just like 'ftp' ). the question is, can a client already operate on incomplete datasets, or must it be fully available at start?

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,525
Credit: 281,477,356
RAC: 46,642

RE: RE: One point made at

Message 58179 in response to message 58178

Quote:
Quote:
One point made at the AEI presentation by Bruce Allen was to keep the crunching parameters ( file sizes etc ) suitable for the lower speed connections around the world. I gather broadband is NOT the majority mode of communication but dial-up IS.

hmmm. how about some kind of "poll", asking users what bandwidth they can assure? or, i could imagine - thou hast to code it! ;) - some preference/setting in the client "prefer BIG chunks" / "rather like it small and handy"

i mean, after some time it would be clear were to put focus on rather than optimising peripherial questions...
though, i do not know how deep this approach of reducing file size impacts on the whole math behind.

one thing doesn't sound logical too much: why reducing size? you'll have to get/return it anyway at some time, and the retrieving/sending of data can be made asynchronious (once an up-/download fails by connection loss, next time transfers would be resumed... just like 'ftp' ). the question is, can a client already operate on incomplete datasets, or must it be fully available at start?

Interesting points. The whole dataset will be the largest yet. But I see the sense in setting the defaults to suit the weakest links ( no offense intended ) in the chain.

Bernard, any thoughts?

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2,139
Credit: 2,706,867,030
RAC: 1,585,896

RE: Interesting points. The

Message 58180 in response to message 58179

Quote:

Interesting points. The whole dataset will be the largest yet. But I see the sense in setting the defaults to suit the weakest links ( no offense intended ) in the chain.

Bernard, any thoughts?

Cheers, Mike.


At the beginning of the run, the bandwidth bottleneck will be at the users' end - obviously, most significantly in the case of dial-up users.

But towards the end of the S5R1 run, we are seeing a different pattern. As we fill in the gaps for WUs which didn't get finished for whatever reason the first time round, I'm seeing my machines switch from datapack to datapack with increasing frequency - and almost all of them are h1 short WUs, with the biggest downloads and the shortest crunch time. One user yesterday commented on receiving 170MB in, near enough, a single download session.

That's tough for the end-user, but it must be horrendous for the server too - I'd like to see a download traffic graph, which must be growing almost exponentially!

If there was any chance of coding for XUL's "small and handy" option, there could be a double benefit: a preference opt-in for users on slower connections, and a server choice when only a few results need to be completed to finish off a sequence.

jowr
jowr
Joined: 19 Feb 05
Posts: 55
Credit: 1,947,636
RAC: 0

RE: But towards the end of

Message 58181 in response to message 58180

Quote:

But towards the end of the S5R1 run, we are seeing a different pattern. As we fill in the gaps for WUs which didn't get finished for whatever reason the first time round, I'm seeing my machines switch from datapack to datapack with increasing frequency - and almost all of them are h1 short WUs, with the biggest downloads and the shortest crunch time. One user yesterday commented on receiving 170MB in, near enough, a single download session.

This is something I have noticed as well. For example:

As of right now, my machine crunches 2 WUs every hour or so [dual core system]. Every 12 WUs comes with an approximately 17 megabyte file associated with the smaller parts. So every 6 hours, I require what amounts to about 20 megs of data.

Due to the fantastically bad uptime of the project in the last 2 weeks [not bitching. Just saying], as of now I am grabbing the maximum amount of data allowable for my machine - 144 units. That is approximately 3.5 days of of crunch time. So right now my client - just my dual core client, I have three other machines - requires about 70 megs of data per day transferred to me to keep up with the processing pace.

I too would love to see a plot of bandwidth and traffic for the server[s? Not sure how E@H is structured in the back end]. I think the technical term is a "metric shitload" of data.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,525
Credit: 281,477,356
RAC: 46,642

RE: At the beginning of the

Message 58182 in response to message 58180

Quote:

At the beginning of the run, the bandwidth bottleneck will be at the users' end - obviously, most significantly in the case of dial-up users.

But towards the end of the S5R1 run, we are seeing a different pattern. As we fill in the gaps for WUs which didn't get finished for whatever reason the first time round, I'm seeing my machines switch from datapack to datapack with increasing frequency - and almost all of them are h1 short WUs, with the biggest downloads and the shortest crunch time. One user yesterday commented on receiving 170MB in, near enough, a single download session.


Wince... and let's be devil's advocates and say that was to simply finalise a single stray shorty unit!

Quote:
That's tough for the end-user, but it must be horrendous for the server too - I'd like to see a download traffic graph, which must be growing almost exponentially!


It would be interesting to see what other evidence could point to the hypothesis that such end of run thrashing ( reminds me of virtual memory and swap files ) did materially contribute to the failures. Certainly plausible.

Quote:
If there was any chance of coding for XUL's "small and handy" option, there could be a double benefit: a preference opt-in for users on slower connections, and a server choice when only a few results need to be completed to finish off a sequence.


Touche!! But I hear the stomping footfalls of the dev's!! :-)

Cheers, Mike

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

F. Prefect
F. Prefect
Joined: 7 Nov 05
Posts: 135
Credit: 1,016,868
RAC: 0

RE: RE: After reading all

Message 58183 in response to message 58175

Quote:
Quote:
After reading all this, I'm still not clear on the future. We're soon to complete S5, right?

S5 data collection at the detectors finishes ~ the middle of this calendar year. The first S5 data analysis, 'S5R1' if you like, finishes in ~ a month ( perhaps a bit longer given recent server troubles ).

There isn't a one-to-one relationship between data collection and analysis because of the many different things that could be searched for. We'll be working on new/enlarged data sets at E@H soon - the software details, data sets etc are being hammered into shape at present. One point made at the AEI presentation by Bruce Allen was to keep the crunching parameters ( file sizes etc ) suitable for the lower speed connections around the world. I gather broadband is NOT the majority mode of communication but dial-up IS.

Quote:
Are we soon to also run out of crunching work for E@H?

No, plenty of science yet.

Quote:
Will there be any crunching down-time for us?

No, servers willing.

Quote:
And if so, when and for how long?

Not-applicable.

Just imagine Dirk Hartog ( or insert your favourite sea-farer here ) returning with a huge list of depth soundings, times/dates, latitudes/longitudes from a round the world trip ..... it'll take a while to do the map and deduce the geography. No shortage of cartography work .... :-)

Cheers, Mike.


As someone who CAN'T, as of yet get any type of broadband, I would echo the above comments regarding dialup as still being the most common means of accessing the web. The largest downloads I have had to date have been 31MB and as I can best recall that took the better of 3 1/2 hrs. Managable, but far from being convenient.

F. Prefedt

In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move.....Douglas Adams

tullio
tullio
Joined: 22 Jan 05
Posts: 2,118
Credit: 61,407,735
RAC: 0

RE: As someone who CAN'T,

Message 58184 in response to message 58183

Quote:


As someone who CAN'T, as of yet get any type of broadband, I would echo the above comments regarding dialup as still being the most common means of accessing the web. The largest downloads I have had to date have been 31MB and as I can best recall that took the better of 3 1/2 hrs. Managable, but far from being convenient.

F. Prefedt


I have a 4 Mbit/s ADSL connection and have downloaded several SuSE Linux releases of about 3.6 GB each. But, since I use BitTorrent with an Azureus client. it may take also days.
Tullio

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.