Can Einstein@Home pass the 1 Petaflop (1000 Teraflop) barrier?

The computing power of Einstein@Home has exceeded 950 Teraflops for the first time since the project was begun in 2005. Based on the rate that our computing power has been growing, I am hopeful that Einstein@Home will pass the 1 Petaflop barrier before the end of 2012. Einstein@Home volunteers: please keep your computers running over the holiday season, and please sign up any new ones that you might receive as a gift!

Bruce Allen
Director, Einstein@Home

Comments

hektorius
hektorius
Joined: 13 Jan 09
Posts: 1
Credit: 1034886
RAC: 0

Can Einstein@Home pass the 1 Petaflop (1000 Teraflop) barrier?

This may be hard since SETI is back in action and shares GPU-s :(

Mayby another power failure .... ;)

Jonatan
Jonatan
Joined: 20 Jun 10
Posts: 66
Credit: 25782906
RAC: 0

We are working hard, in our

We are working hard, in our challenge...

December maximum power is trying to increase the power of einstein@home and also try to finish the S6LV1 before new year...

But is very difficult because good computers compute bigger GPU data than CPU...We are crunching because is hard to get...

Greetings from Spain :))

Russell
Russell
Joined: 1 Oct 11
Posts: 6
Credit: 14411295
RAC: 0

Everyone has a friend or

Everyone has a friend or family member somewhere that might be interested in volunteering their computing resources.

I like to think my desktop computer helps keep my bedroom warm over the winter months as it crunches :)!

juan BFP
juan BFP
Joined: 18 Nov 11
Posts: 839
Credit: 421443712
RAC: 0

Os little team adds another

Os little team adds another 4MM credits to the E@H cause today... 1 PF mark is a little closer.

lHj2ixL.jpg

 

mmstick
mmstick
Joined: 6 Jun 12
Posts: 14
Credit: 2066411
RAC: 0

Adding a Radeon HD 7950 and

Adding a Radeon HD 7950 and two Radeon HD 6850s to help you reach that goal.

Sebastian
Sebastian
Joined: 6 Mar 12
Posts: 4
Credit: 1681766
RAC: 0

Sorry I can't keep my pc

Sorry I can't keep my pc running over the holiday season.

Mark W. Patton
Mark W. Patton
Joined: 2 May 09
Posts: 19
Credit: 105147736
RAC: 0

Hello Bruce I am as excited

Hello Bruce
I am as excited as you that the project has accomplished so much. If I had the money, I would place more resources in operation to do more work. I can assure you though, that I intend to keep the resources I do have working 24/7, as they do all the time. I think we are going to make the goal by the end of the year.
Regards,
Mark W. Patton

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: We are working hard, in

Quote:

We are working hard, in our challenge...

December maximum power is trying to increase the power of einstein@home and also try to finish the S6LV1 before new year...

But is very difficult because good computers compute bigger GPU data than CPU...We are crunching because is hard to get...

Greetings from Spain :))

Greetings and thank you!

Bruce

Director, Einstein@Home

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: Everyone has a friend

Quote:

Everyone has a friend or family member somewhere that might be interested in volunteering their computing resources.

I like to think my desktop computer helps keep my bedroom warm over the winter months as it crunches :)!

Thank you for your support, and remember, if it gets too hot in the bedroom, you can always open the window (:-).

Bruce

Director, Einstein@Home

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: Os little team adds

Quote:
Os little team adds another 4MM credits to the E@H cause today... 1 PF mark is a little closer.

Thank you!

Bruce

Director, Einstein@Home

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: Adding a Radeon HD 7950

Quote:
Adding a Radeon HD 7950 and two Radeon HD 6850s to help you reach that goal.

That's the spirit!!!

Bruce

Director, Einstein@Home

juan BFP
juan BFP
Joined: 18 Nov 11
Posts: 839
Credit: 421443712
RAC: 0

RE: RE: Os little team

Quote:
Quote:
Os little team adds another 4MM credits to the E@H cause today... 1 PF mark is a little closer.

Thank you!

Bruce


We will try to do our best and repeat/pass that mark today!

If only i know how to make the 590/690´s use more than 50% of it´s real power... as is easy to do with his cousins 560/580/670/680... 5MM will be a easy task... SNIF!!!

But still planning how to do that in the drawing board!!! Any help is helcome! (Yes, I allready increase the BRP factor...)

lHj2ixL.jpg

 

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: Hello Bruce I am as

Quote:
Hello Bruce
I am as excited as you that the project has accomplished so much. If I had the money, I would place more resources in operation to do more work. I can assure you though, that I intend to keep the resources I do have working 24/7, as they do all the time. I think we are going to make the goal by the end of the year.
Regards,
Mark W. Patton

Mark, thank you for the kind words, and for your support!

Bruce

Director, Einstein@Home

Sebastian
Sebastian
Joined: 6 Mar 12
Posts: 4
Credit: 1681766
RAC: 0

Do you have any idea Bruce

Do you have any idea Bruce what we will compute after finishing S6LV1 ? Are there any other project ideas beside the BRP4 Arecibo pulsar search? And maybe you or anyone else can answer one more question: Is seti@home still doing serious scentific analysis on their data like einstein@home does?

Thanks for any answers, Merry Christmas and a happy new year!
Greetings from Munich, Germany.

Sebastian
Sebastian
Joined: 6 Mar 12
Posts: 4
Credit: 1681766
RAC: 0

I have been to the LIGO

I have been to the LIGO Observatory in Hanford this summer (great tour btw!), and it looks like It is going to take a long time until we get new data.

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: Do you have any idea

Quote:

Do you have any idea Bruce what we will compute after finishing S6LV1 ? Are there any other project ideas beside the BRP4 Arecibo pulsar search? And maybe you or anyone else can answer one more question: Is seti@home still doing serious scentific analysis on their data like einstein@home does?

Thanks for any answers, Merry Christmas and a happy new year!
Greetings from Munich, Germany.

The Einstein@Home project is currently testing an extension of S6LV1 to higher frequencies. Hopefully coming soon to a computer near you!

We'll continue with the BRP4 search indefinitely. First we need to catch up with the data backlog of about 13,000 beams (this will take a couple of months at the current rates) then we'll continue in the steady-state. PALFA is taking about 50 new beams of data per day, on the average, so this search will be running for some years, at least for as long as PALFA is collecting new data.

I don't have any inside knowledge of the SETI@Home project, but have always thought that the basic idea is outstanding. This is high-risk/high-value research, in the sense that the odds of success are not high, but the payoff is huge.

I personally am entirely sure that there is other life in our universe, but finding definite proof of this would be one of the most important discoveries ever for mankind. So personally I would be very happy to see SETI@home succeed in its goals.

Happy Holidays, and thank you for supporting Einstein@Home!

Bruce

Director, Einstein@Home

David S
David S
Joined: 6 Dec 05
Posts: 2473
Credit: 22936222
RAC: 0

I just added a cheap GPU to

I just added a cheap GPU to one of my less-capable crunchers. However, I set up new prefs for it to do GPU work only, and then I set it to no new tasks until it finishes all the CPU work it has on board. There is only one Einstein unit left, but a bunch of Seti. It should finish that up in a week or so and then I'll unleash the GPU for both projects.

David

Miserable old git
Patiently waiting for the asteroid with my name on it.

GALAXY-VOYAGER
GALAXY-VOYAGER
Joined: 24 Oct 12
Posts: 32
Credit: 92959
RAC: 0

Fingers Crossed that we can

Fingers Crossed that we can Achieve That Goal. My 2 Computers are almost Running 24/7. I just put them in Hibernate or Sleep Mode for a few hours or overnight, each couple of days, just to give them a rest. I only Shut Down about once or twice a week. I'm not expecting any New Ones at Xmas, but hopefully will be getting my Old Computer Working Again, and if I do, I'll be Activating it as soon as I can. I currently Run 5 Projects on My Desktop: Seti@Home, Einstein@Home, Cosmology@Home, Rosetta@Home, Milkyway@Home. I also have Orbit@Home, but have not done any work as yet, due to its Inactivity. On my Notebook (Laptop), I only Run Seti@Home due to it only having 2GB RAM and if I Run too many things on it, everything either Slows Down or Crashes. Even without any Programs Open, The RAM Usage is between 60 and 90 Percent. I do quite a few Einstein Tasks on my Desktop, and My Laptop is always loaded with Seti@Home tasks (I have had 35 Seti Tasks on it at one time). I actually got involved in The (Other) BOING Projects through The S.E.T.I. Program.

Yan Novoprudskiy
Yan Novoprudskiy
Joined: 12 Sep 12
Posts: 2
Credit: 4391671
RAC: 0

Today Ill add another 24/7

Today Ill add another 24/7 machine with Kepler GPU

Vegard
Vegard
Joined: 2 Sep 11
Posts: 8
Credit: 13739271
RAC: 0

Hi Bruce, could you give

Hi Bruce,

could you give us some information about the plans for publications you have for the already processed data? Will there be publications about the gamma-ray searches? And what about the S6 search? The computations ended almost a year ago or so and I am wondering, if there will be a similar paper as for the S5 search that was published a few month ago. And what about the pulsars found in Arecibo data? Will there be a publication?

Cheers,

Vegard

Nathan Fandrich
Nathan Fandrich
Joined: 14 Mar 12
Posts: 1
Credit: 2066945
RAC: 0

Suspended other causes to

Suspended other causes to help achieve 1 petaflop goal. I should be getting a desk top soon. Will add as soon as I do. You people are doing great work and I wish others would join. I will also put it up on facebook.

MusicGod
MusicGod
Joined: 17 Jul 05
Posts: 1
Credit: 4132841
RAC: 0

Seti@Home has been giving

Seti@Home has been giving fits over the past month ...and I know as an Einstein cruncher who had his project set for "no new tasks" while crunching SETI has returned full time after 3 years off and crunching high numbers. I`m sure other Seti crunchers just got fed up with the condiciton of the project and left, thus returning to Einstein@hoime like I did./...good luck and I hope yo umake it.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6591
Credit: 323967168
RAC: 174520

After fossicking about I've

After fossicking about I've rustled up a couple of i7 machines with NVidia 560+ cards - they can do some fine E@H work when my kids aren't online gaming ( they have to sleep sometime ) - and I reckon I may find some more .... :-) :-)

[ BTW : if you have a Linux machine may I recommend using the 'per distribution' method of installing BOINC, it does an end-run around the silly business with the later Ubuntu's re. logging on as root ]

Cheers, Mike.

( edit ) As for heat ( summer DownUnda ) : I have found some great water cooling units that are sealed from Corsair ( H80i for ~ $140 AUD ) that drop the CPU cores down to ~ 25C. Big & quiet fans. Choice of brackets to mount to the various CPU form factors. Heat exchange right over the chip. No trouble whatsoever.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

archae86
archae86
Joined: 6 Dec 05
Posts: 3161
Credit: 7264678432
RAC: 1568975

RE: I have found some great

Quote:
I have found some great water cooling units that are sealed from Corsair ( H80i for ~ $140 AUD ) that drop the CPU cores down to ~ 25C. Big & quiet fans. Choice of brackets to mount to the various CPU form factors. Heat exchange right over the chip. No trouble whatsoever.


Mike, those fans are listed as 120mm 2700 rpm. A pair that size running that fast would not be welcome companions in my study. What rpm do you find yourself actually running? And how do you find the fan noise?

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6591
Credit: 323967168
RAC: 174520

RE: RE: I have found some

Quote:
Quote:
I have found some great water cooling units that are sealed from Corsair ( H80i for ~ $140 AUD ) that drop the CPU cores down to ~ 25C. Big & quiet fans. Choice of brackets to mount to the various CPU form factors. Heat exchange right over the chip. No trouble whatsoever.

Mike, those fans are listed as 120mm 2700 rpm. A pair that size running that fast would not be welcome companions in my study. What rpm do you find yourself actually running? And how do you find the fan noise?


Ah well, the part of the unit that sits on the CPU has a large push button on it ( or later models have an on-screen widget via USB if you like ) to cycle/select different behaviours for the entire assembly. From 'gentle' to 'aggressive' response, determining the fan speeds and the coolant pump rate. So the 2700 rpm is the max for that. I set it to gentle and for me it occasionally revs up but not for long.

[ One can reduce the activity of the case fans on account of having the CPU cooler active, so there's a dB gain there too. ]

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

juan BFP
juan BFP
Joined: 18 Nov 11
Posts: 839
Credit: 421443712
RAC: 0

Another 4MM (3,991,963) added

Another 4MM (3,991,963) added to the E@H cause yesterday but the total flops still at 943.4 TFLOPS. Did we reach the actual total capacity?

lHj2ixL.jpg

 

nhk86
nhk86
Joined: 12 Jan 12
Posts: 1
Credit: 25684
RAC: 0

bonsoir j'arrête pour le

bonsoir
j'arrête pour le moment les autres projets et je met toute ma puissance disponible sur Einstein@home pour passer à 1 pétaflop!!!!!
bonne réception

dskagcommunity
dskagcommunity
Joined: 16 Mar 11
Posts: 89
Credit: 1219701683
RAC: 53907

And this means in english?..

And this means in english?..

DSKAG Austria Research Team: [LINK]http://www.research.dskag.at[/LINK]

transient
transient
Joined: 3 Jun 05
Posts: 62
Credit: 115835369
RAC: 0

My french is not that good,

My french is not that good, but it means something like: I'll stop crunching for my other projects and will focus on Einstein.

Jord
Joined: 26 Jan 05
Posts: 2952
Credit: 5893653
RAC: 5

RE: The computing power of

Quote:
The computing power of Einstein@Home has exceeded 950 Teraflops for the first time since the project was begun in 2005.


And now that you pointed that out, the project immediately got scared and the TFLOP amount is in decline ever since. ;-)

Erwin_der_Erste
Erwin_der_Erste
Joined: 27 Apr 07
Posts: 1
Credit: 7852746
RAC: 0

Why should Einstein@Home pass

Why should Einstein@Home pass the 1 Petaflop (1000 Teraflop) barrier this time?
If I understand right the S6LV1 search progress is almost completed and in addition to this the sensitivity of interferometers has to be increased before creating new gravitation wave work units?

Best regards.

Markus

juan BFP
juan BFP
Joined: 18 Nov 11
Posts: 839
Credit: 421443712
RAC: 0

Another >4MM day (4,210,026)

Another >4MM day (4,210,026) our team dialy record, a 20.1 Tflop added to the E@H pot!

lHj2ixL.jpg

 

ongjenyung
ongjenyung
Joined: 19 Apr 10
Posts: 5
Credit: 29073393
RAC: 0

预ç¥çˆ±å› æ–¯å¦@home项目çš

预ç¥çˆ±å› æ–¯å¦@home项目的计算能力çªç ´ä¸€åƒä¸‡äº¿æ¬¡å¤§å…³ã€‚

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6591
Credit: 323967168
RAC: 174520

RE: 预ç¥çˆ±å› æ–¯å¦@homeé¡

Quote:
预ç¥çˆ±å› æ–¯å¦@home项目的计算能力çªç ´ä¸€åƒä¸‡äº¿æ¬¡å¤§å…³ã€‚


Which via Google translate:

"I wish the computing power of the Einstein @ home project exceeded $ 11 trillion mark."

Thanks for that! :-)

Cheers, Mike.

( edit ) Adjusting the translation in context : $ 11 trillion = 1 Petaflop :-)

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 758450897
RAC: 1145481

RE: Why should

Quote:

Why should Einstein@Home pass the 1 Petaflop (1000 Teraflop) barrier this time?
If I understand right the S6LV1 search progress is almost completed and in addition to this the sensitivity of interferometers has to be increased before creating new gravitation wave work units?

Best regards.

Markus

Note quite. The current run will be extended to cover higher spin frequencies, and after that is done we are planning to have a new search, using different methods but not waiting for the detector upgrades. See this thread for updates:
http://einsteinathome.org/node/196604.

Cheers
HB

WezH
WezH
Joined: 4 May 07
Posts: 8
Credit: 55446727
RAC: 0

RE: And now that you

Quote:
And now that you pointed that out, the project immediately got scared and the TFLOP amount is in decline ever since. ;-)

And will decline more when December maximum power -team challenge is over. I believe that most of GPUUG's members will switch back to S@H

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 758450897
RAC: 1145481

RE: RE: And now that you

Quote:
Quote:
And now that you pointed that out, the project immediately got scared and the TFLOP amount is in decline ever since. ;-)

And will decline more when December maximum power -team challenge is over. I believe that most of GPUUG's members will switch back to S@H

There's also BOINC's "debt" system which will have an effect: while SETI@Home was not able to distribute new work, many hosts participating in both S@H and E@H have received more work (and later credits) from E@H than they usually would according to the chosen resource share. BOINC tries to correct this in the long term by throttling the work fetch from E@H until the "debt" is repaid. Fair enough.

Cheers
HB

eeqmc2_52
eeqmc2_52
Joined: 10 May 05
Posts: 38
Credit: 3714564606
RAC: 1167609

Real men run 24/7/365, year

Real men run 24/7/365, year in, year out. No holidays or vacations, just brief downtime for hardware upgrades. Time to brass-up and hit 1 Petaflop. Your PC can holiday when you're dead.

There are only 10 kind of people in the world, those that understand binary and those that don't!

transient
transient
Joined: 3 Jun 05
Posts: 62
Credit: 115835369
RAC: 0

RE: Real men run 24/7/365,

Quote:
Real men run 24/7/365, year in, year out. No holidays or vacations, just brief downtime for hardware upgrades. Time to brass-up and hit 1 Petaflop. Your PC can holiday when you're dead.

I understand you're about to offer to pay my electricity bill for me? LOL

Gatekeeper
Gatekeeper
Joined: 23 Jun 10
Posts: 11
Credit: 6898641
RAC: 0

RE: RE: Real men run

Quote:
Quote:
Real men run 24/7/365, year in, year out. No holidays or vacations, just brief downtime for hardware upgrades. Time to brass-up and hit 1 Petaflop. Your PC can holiday when you're dead.

I understand you're about to offer to pay my electricity bill for me? LOL

Depending on where you live, your computer might provide a cheaper heat source than your furnace. :)

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 0

Congratulations on almost

Congratulations on almost reaching 1 Petaflop!

I wish I could do more to help the project. I have the hardware to run about 1200 BRP4 tasks and additional S6LV1 tasks per day but not the bandwidth/throughput to keep all my systems crunching full time. My hosts seem unable to download at a rate of more than 1.4 Mbps per host from the Einstein download servers. Since I have a limited window each day of unrestricted bandwidth, it has been difficult to keep the systems fed with work.

Neil Newell
Neil Newell
Joined: 20 Nov 12
Posts: 176
Credit: 169699457
RAC: 0

RE: Congratulations on

Quote:

Congratulations on almost reaching 1 Petaflop!

I wish I could do more to help the project. I have the hardware to run about 1200 BRP4 tasks and additional S6LV1 tasks per day but not the bandwidth/throughput to keep all my systems crunching full time. My hosts seem unable to download at a rate of more than 1.4 Mbps per host from the Einstein download servers. Since I have a limited window each day of unrestricted bandwidth, it has been difficult to keep the systems fed with work.

That made me wonder how compressible the data is, so I tried bzip2 compression on a couple of files:-

p2030.20120101.G177.93-02.24.N.b1s0g0.00000_543.bin4 2098320 bytes in=> 450585 out
h1_0403.95_S6GC1 8445248 bytes in => 7130498 bytes out

So based on this limited sample, it appears BRP4 files are pretty compressible (almost 5x) but SL6V1 less so (<1.2x). If it's possible to somehow arrange compressed downloads for you, you could get almost 5 times the work in the same bandwidth.

Is it possible to proxy e@h downloads? I think it downloads with wget or curl, so in principle it seems possible to insert a compress/decompress step.

edit: compressing about 20 BRP4 files gave an average compression of ~3.58x; files fell into 3 roughly similiar sized groups, with compression rations of 4.72x, 3.73x and 2.84x)

J
J
Joined: 22 Sep 12
Posts: 1
Credit: 9288481
RAC: 12194

Merry Christmas Bruce. I

Merry Christmas Bruce. I will leave my computer on for the rest of the year. Thank you for your commitment to the advancement of the understanding of nature.

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 0

RE: That made me wonder

Quote:


That made me wonder how compressible the data is, so I tried bzip2 compression on a couple of files:-

p2030.20120101.G177.93-02.24.N.b1s0g0.00000_543.bin4 2098320 bytes in=> 450585 out
h1_0403.95_S6GC1 8445248 bytes in => 7130498 bytes out

So based on this limited sample, it appears BRP4 files are pretty compressible (almost 5x) but SL6V1 less so (<1.2x). If it's possible to somehow arrange compressed downloads for you, you could get almost 5 times the work in the same bandwidth.

Is it possible to proxy e@h downloads? I think it downloads with wget or curl, so in principle it seems possible to insert a compress/decompress step.

edit: compressing about 20 BRP4 files gave an average compression of ~3.58x; files fell into 3 roughly similiar sized groups, with compression rations of 4.72x, 3.73x and 2.84x)

I think that would be fantastic if there was a way to compress the data down in BOINC either with Bzip2 or possibly LZMA2 (xz). The compression ratios that you are seeing with BRP4 files are excellent. The bandwidth is the main item that has kept me from bringing more resources to the project. The previous reduction from 32 MB to 16 MB per task made by the project team was very helpful. However, even at 16 MB per task, I am looking at almost 19 GB of data transfer per day to bring all my resources online full time.

Jord
Joined: 26 Jan 05
Posts: 2952
Credit: 5893653
RAC: 5

RE: I think that would be

Quote:
I think that would be fantastic if there was a way to compress the data down in BOINC either with Bzip2 or possibly LZMA2 (xz).


From [trac]wiki:FileCompression[/trac]: The BOINC client can handle compressed downloads using both the mod_gzip and mod_default mechanisms provided by Apache. However, resumption of interrupted downloads doesn't work with either of these methods, so we don't recommend using them.

For application version files, you can specify a flag. This will cause the file to be transferred in compressed form to 7.0+ clients.

and

If you include the tag in an output file description, the file will be gzip-compressed after it has been generated.

The gzip_when_done is only supported in client version 5.8+.
(ClimatePrediction uses this form)

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 0

RE: RE: I think that

Quote:
Quote:
I think that would be fantastic if there was a way to compress the data down in BOINC either with Bzip2 or possibly LZMA2 (xz).

From [trac]wiki:FileCompression[/trac]: The BOINC client can handle compressed downloads using both the mod_gzip and mod_default mechanisms provided by Apache. However, resumption of interrupted downloads doesn't work with either of these methods, so we don't recommend using them.

For application version files, you can specify a flag. This will cause the file to be transferred in compressed form to 7.0+ clients.

and

If you include the tag in an output file description, the file will be gzip-compressed after it has been generated.

The gzip_when_done is only supported in client version 5.8+.
(ClimatePrediction uses this form)

Thanks for the information.

I wonder if it would be possible to have the BRP4 application itself link to one of the different available compression libraries and have the application handle decompression of the data files before starting processing of the data.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4330
Credit: 251555871
RAC: 37253

Thank you for looking into

Thank you for looking into this. I did some compression experiments with the LIGO data (currently used in S6LV1), but not with the BRP data (at least not since ABP days).

Quote:
I wonder if it would be possible to have the BRP4 application itself link to one of the different available compression libraries and have the application handle decompression of the data files before starting processing of the data.

This is what I'm planning to do: Link the application with zlib, so it can easily handle both gzipped and plain data, then add gzip as the last step of pre-processing. Shouldn't require much of a change.

We are already using the gzip feature of the BOINC Client for result files in BRP and GW work (), in fact this is why Einstein@Home requires a minimum Client version of 5.8. But I wouldn't rely on that feature for input/workunit files.

BM

BM

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 0

RE: This is what I'm

Quote:


This is what I'm planning to do: Link the application with zlib, so it can easily handle both gzipped and plain data, then add gzip as the last step of pre-processing. Shouldn't require much of a change.

We are already using the gzip feature of the BOINC Client for result files in BRP and GW work (done>), in fact this is why Einstein@Home requires a minimum Client version of 5.8. But I wouldn't rely on that feature for input/workunit files.

BM

I did some testing last night and compressed a few of the .bin4 files with gzip. The files compressed from 2052K to 464-468K. This is a substantial reduction in file size.

I think that linking the application to zlib would be very helpful. Thank you.

Filipe
Filipe
Joined: 10 Mar 05
Posts: 186
Credit: 411419884
RAC: 189390

RE: We'll continue with the

Quote:
We'll continue with the BRP4 search indefinitely. First we need to catch up with the data backlog of about 13,000 beams (this will take a couple of months at the current rates) then we'll continue in the steady-state. PALFA is taking about 50 new beams of data per day, on the average, so this search will be running for some years, at least for as long as PALFA is collecting new data.

Since we are processing an average of 166beams/day, does that mean that we are going to have a shortage of BRP4 tasks to feed all the GPU´s?

eeqmc2_52
eeqmc2_52
Joined: 10 May 05
Posts: 38
Credit: 3714564606
RAC: 1167609

I figure my home computing

I figure my home computing taps me for 50 cents per day maximum. I'll balance my cost by having a glass of water rather than a can of diet coke.

There are only 10 kind of people in the world, those that understand binary and those that don't!