Information about the new S5 workunits
15 Jun 2006 4:38:48 UTC
Topic 191381
(moderation:
I'll update this as our generation and testing of our first S5 workunits proceeds
The names of these workunits are of the form h1_XXXX.X_S5R1x_* and l1_XXXX.X_S5R1x_*.
The application running these is called einstein_S5R1. This application incorporates many of the speedups and other code changes suggested by Akos. Note: please do NOT replace these stock executables with custom versions of 'albert'. They are not compatible.
There are two types of workunits: short and long. The short workunits have XXXX.X less than or equal to 0400.0.
There are also two types of data files: short and long. The short data files (l1_XXXX.X) are from the LIGO Livingston Observatory, and are about 4.5MB in size. The long data files (h1_XXXX.X) are from LIGO Hanford and are about 16MB in size. Note: once your computer downloads one of these data files, it should be able to do many workunits for that same file.
We are switching to a new uniform system for awarding credits. All users on all platforms will claim the same credit for each workunit, with an amount of credit proportional to the length of the workunit: 'equal credit for equal work'.
To try and increase the total amount of computing power available to the project, we have changed the target number of results and minimum quorum from 3 to 2. Only if the first two results from different hosts/users do not agree will additional work be generated.
Please be patient with us if we have to sort out last minute problems or other issues. We have been testing this privately for some time, so we are fairly confident that there are no significant issues that remain. Nevertheless, several of us on 'the cutting edge' are rather short on sleep!
Bruce
Added 14/06/2006: please feel free to post questions here, but keep in mind that it may take some time before we have time to answer everything.
Added 15/06/2006: we have now generated about 1500 workunits. So far all is looking well. There are a fairly large number of download errors in downloading data files. But these appear to be mostly due to some problems we had yesterday in replicating data to our mirror sites. There are also interesting errors on *some* Mac OS X PPC systems. I'm sure we'll sort this out quickly.
Added 16/06/2006: Bernd Machenschalk has tracked down and fixed the error refered to above. It only occured on Mac OS X PPC G3 systems. We will be distributing a new application later today. We are also going to now finish creating the last of the remaining S4 workunits! On the project status page, you will now see that the S4 workunit generation is complete (but that the number of unsent results will be quite high for a couple of days).
Added 16/06/2006: We have completed the GENERATION of the remaining S4 workunits. They are in the database and should complete crunching during the next 2 to 3 weeks. We have also begun automatic generation of the S5 workunits, and have updated the server status page to reflect this. I also added an item on this status page to track the progress of the remaining S4 workunits.
Added 20/06/2006: The transition to S5 workunits seems to be proceeding very smoothly. There are currently only about 13,000 remaining (unsent) S4 workunits in the database; the remainder are either in progress or finished. There are now in excess of 250,000 S5 workunits in progress and the S5 failure rate is well under 5%. So from the project perspective the overall picture is extremely positive: the S4 -> S5 transition is essentially complete and was successful. In a few more days when the S5 work in progress is dominant (with respect to S4 work) I will do some more modifications of the server_status page to estimate the S5 analysis completion date.
Director, Einstein@Home
Information about the new S5 workunits
)
Thanks for the update Bruce! The new credit system and quorum change sounds excellent to me. My appreciation to all involved in the project.
Thank you Bruce, we highly
)
Thank you Bruce, we highly appreciate that you keep providing us with up-to-date information, since for many participants it is important to know what they are crunching.
It's amazing to be part of this project and to know that now we've shifted to two times more precise data!
@nevermorestr: the banana emoticon is cool! :-)
It might be worth to
)
It might be worth to add:
- the analysis code in the new Apps is the same that's used in the current Beta Test Apps, so the S5R1 Apps should be precisely as fast as them. However the current Beta Test Apps won't work as they are now, you will have to remove the app_info.xml file to get the new official Apps and work for S5R1.
- To make up for the faster Apps we increased the size of the workunits. The "long" ones will be roughly five times as long as the "long" ones from S4, the "short" ones will be roughly twice as long as their S4 counterparts.
BM
BM
Thanks for the update Bruce.
)
Thanks for the update Bruce.
I hope the credit change does not cause the issues it has over at SETI. Sounds like what you are doing is tied more to the size of the WU, and not how long it takes. Or at least thats what I am taking from it. If I am wrong please correct me.
Come Join us at Hawaiian Beach Bums
Yes, thanks for keeping the
)
Yes, thanks for keeping the troops informed.
So what is the right way to transition to the new S5 work if currently running a beta app?(Linux 4.58) Will just removing the xml work, or should I set no new work and let the beta run dry, then delete the xml???
Edith says: problem handled.
)
Edith says: problem handled.
RE: Yes, thanks for keeping
)
For now just wait. We're only distributing very few S5R1 WUs right now for some fine-tuning. When we start distributing them on large scale, we will probably announce it here (and/or on the front page). Then both ways should work. Your second approach, i.e. setting "no new work" first, is, however, the cleanest way, so preferable.
Note that while S4 units are still around, you may also get them after you removed the app_info.xml and you will download and run them with the old official App (4.40 in your case). We can't completely avoid this, but the way to make this rather unlikely would be to reset the project after your client has ran out of S4 work to remove the data files that still refer to S4.
BM
BM
RE: I hope the credit
)
I'm not aware of what's going on at SETI, and all project people, I think, have their hands full with E@H. Can you give me a short summary (without initiating a new discussion here) what the trouble or different opinions are there @SETI?
The credit will be granted totally based on the size of the Workunits as determined on the server side, regardless of the time it takes a specific host or App to process it. Maesurements have been incorporated into the Apps so that they should also claim this credit for transparency, but the credit actually granted this way doesn't need to have anythig to do with the claimed credit anymore.
BM
BM
Do I need Boinc-client 5.4.9
)
Do I need Boinc-client 5.4.9 or is a lower version also possible?
RE: I'm not aware of what's
)
The credit issues was a big problem, as was the fact the application was rushed and had some serious bugs still in it. They lengthened the units with a credit change at the same time. So the credit issued did a nose dive to about 1/3 what it was. This credit dive caused serious issues with the people that compete with them. This has hurt the project on total crunchers that left due to the issues.
As far as the Application, it had some serious hang and loop issues that really had people bent out of shape. WU's that would run and get to finish, then restart and run again, things of that nature. They also did a complete stop of the old WU's and issued only new WU's with no real blend over period. So when the Bug issues came up, many crunchers had no way to keep going. Many got mad about the whole issue and left completly.
It seems the credit system you are changing to may be a better solution than what SETI went to. Which was a flop count method. It seems to be in step with the longer WU's. I hope all goes much smoother here. If it does, it may be a model for SETI to change too.
Come Join us at Hawaiian Beach Bums