It's good to know there's a "market" for more of a progress meter. Some of it has already been in place for a while.
If you haven't looked at the "Server Status" link on the main page, do so now. The most interesting number to the people on this thread is probably at the bottom of the middle column, titled "floating point speed in last 7 days". That's a measure of the real computing power you all have been putting in lately. Right now it's a factor of 3 down from the maximum number because not all the hosts are running. I don't know how this compares to other distributed computing projects, but I gather it's universal that some people sign up but don't run it or (like me) stop it on an older machine after a while.
I didn't ask for a progress meter, I asked for a progress report. I appreciate you trying to understand what we are asking for, but I think you may be missing the point. If you re-read my post, that got fairly positive feedback, it should be pretty obvious that "floating point speed in last 7 days"-number isn't exactly what I was asking for.
You really need to improve the way you communicate with people who are donating their CPU cycles if you want to ensure their continued support.
Bruce is working on a much more graphic and interesting summary of the S3 analysis, including measures of scientific interest (and descriptions of how the science is actually done).
This sort of scientific article is very welcome indeed, but I suspect that the low frequency that you will be able to make these means you will need to take other steps to keep people interested.
Jan,
Don't forget - These people running the project don't owe us anything. Yes, we are generously letting them use our computers to run research programs, but we can't get involved out of interest and then demand something in return! That said, it's of course appreciated when we get more details on what is going on. I just think your post, Jan, had a bit too much of a demanding tone to it.
You imply that the people at Einstein@Home have to do more to keep people running the program.....well, I say that people who were interested enough to sign up for the project and download BOINC in the first place are probably not going to up and leave over some lack of detailed status reports.....
You imply that the people at Einstein@Home have to do more to keep people running the program.....well, I say that people who were interested enough to sign up for the project and download BOINC in the first place are probably not going to up and leave over some lack of detailed status reports.....
Nonetheless, I find my interest is reinvigorated whenever a project scientist responds to a question or gives a brief update. For example, for a well communicated and scientifically significant project such as Folding@home, I stayed with for over three years.
The beauty of a project like Einstein is that it will produce significant scientific results, and enhance in a fundamental way our understanding of the universe. And the results will be published and available for all to see. Plus the project has its act together in terms of the solidness of its servers and its general operation.
The numbers for total work units and work units completed are there too. From the numbers that are there you can work out percent completion and estimated completion time.
Hope this helps,
Ben
Ben,
I feel like I should have my head examined! I never looked too closely at the server status page in terms of doing the math to see how the project was progressing. I guess I just assumed that the listed database was only a partial database, because E@h wanted to limit the number of database workunits at any given time.
But I guess I made a wrong assumption!
And it looks like with only about 14,725 Unsent results, this S3 phase of the project is about to come to an end?
Perhaps a sentence or two on the Server Status page explaining what the data means and how one can calculate some basic statistics would be useful.
The numbers for total work units and work units completed are there too. From the numbers that are there you can work out percent completion and estimated completion time.
Hope this helps,
Ben
And it looks like with only about 14,725 Unsent results, this S3 phase of the project is about to come to an end?
Thanks for your wonderful posts! :)
Well - I've seen these numbers go up and down last few weeks.
Don't forget - These people running the project don't owe us anything. Yes, we are generously letting them use our computers to run research programs, but we can't get involved out of interest and then demand something in return! That said, it's of course appreciated when we get more details on what is going on. I just think your post, Jan, had a bit too much of a demanding tone to it.
You imply that the people at Einstein@Home have to do more to keep people running the program.....well, I say that people who were interested enough to sign up for the project and download BOINC in the first place are probably not going to up and leave over some lack of detailed status reports.....
I could pull plug on this project just to prove you wrong, but I won't ;-)
I guess my post was a little whiny and demanding. I can assure you that was not my intention. I can appreciate that project has limited ressources and have to prioritize their efforts.
I wanted to ensure project team understood there is a slight difference between runtime statistics and what I proposed. I guess the "keeping people interested"-part was speculative and not entirely fair way emphasize my points.
Don't forget - These people running the project don't owe us anything. Yes, we are generously letting them use our computers to run research programs, but we can't get involved out of interest and then demand something in return! That said, it's of course appreciated when we get more details on what is going on. I just think your post, Jan, had a bit too much of a demanding tone to it.
You imply that the people at Einstein@Home have to do more to keep people running the program.....well, I say that people who were interested enough to sign up for the project and download BOINC in the first place are probably not going to up and leave over some lack of detailed status reports.....
I could pull plug on this project just to prove you wrong, but I won't ;-)
I guess my post was a little whiny and demanding. I can assure you that was not my intention. I can appreciate that project has limited ressources and have to prioritize their efforts.
I wanted to ensure project team understood there is a slight difference between runtime statistics and what I proposed. I guess the "keeping people interested"-part was speculative and not entirely fair way emphasize my points.
Regards
Jan H. Hansen
I, for one, signed up enthusiatically. Then I got worried when I noticed how hot my laptop got when I generously left in on 24/7 as a contribution to E@H. Now I only leave it on when I am doing something for me (letting E@H use 98% or my cpu.) So although I volunteer for free, I _do_ feel they owe me something in return for the worry I have, if not for the actual risk, of frying my computer before its time. Something more immediate than the abstract knowledge that I am helping science.
There is an old saying that "you get what you pay for." Well, the E@H folks have a very high rate of return (50,472 active users as of this moment: 50K computers -- probably $50,000,000 worth!) for very little cost. What an amazing benefit/cost ratio!! Now if they invested a little more by providing an occasional update, or perhaps even just a jazzier display on the server status page, and maintained only a fraction of their present cost/benefit ratio, then they could get a greater number of active users. Where else can one get so much for so little. This, of course, assumes they want more acive users.
If a person's behavior gets no positive reinforcement, the behavior tends to be extinguished. I used to be at SETI, where there was a nice little display showing what _my_ computer was doing. I could even zone out looking at it, hoping I would be watching when something significant happened. This was all very irrational, since I wouldn't have the faintest idea what to look for. It was a little like playing the lottery for a 1:200,000,000 chance. But it also fed my ego, for this was _my_ work (even though I had nothing at all to do with it -- it was my computer's work product). There was a little add-on that graphically showed me all of and the best of the various foobars I had found. Even though I did not really know what the foobars were, it satisfied my need to feel that I actually accomplished something. The only thing similar in E@H is the screen saver, and while it may be showing my data, there is something monotonous and impersonal about it. It would be nicer to show, for example, what frequency bank I was processing, or some pattern related to the output. Even though I might not know that it meant, I could say, "aha, I got a really big squiggle there in the left hand side this time", and I could dream that I had been a part of finding a real wave.
Please don't forget the human side of this. We are, even the mathematicians, full of irrational human tendencies. You may get more computer time donated if you package the product a little nicer, and pay attention to these irrational qualities that motivate us.
...
There is an old saying that "you get what you pay for." Well, the E@H folks have a very high rate of return (50,472 active users as of this moment: 50K computers -- probably $50,000,000 worth!) for very little cost. What an amazing benefit/cost ratio!! Now if they invested a little more by providing an occasional update, or perhaps even just a jazzier display on the server status page, and maintained only a fraction of their present cost/benefit ratio, then they could get a greater number of active users. Where else can one get so much for so little. This, of course, assumes they want more acive users.
If a person's behavior gets no positive reinforcement, the behavior tends to be extinguished. ...
Thanks,
Tom
Well said, and I agree. I had 2 PCs running E@H. One in the attic that I set up and got running again just for this project. I recently had some in-house LAN problem, took the old PC down "temporarily," and decided not to start it up again.
Why? First of all, at $0.14/kwhr and 130 watts, it costs about $13/month to run the it. Second, I don't really know what "progress is being made." At this point, it's alll very abstract to me -- the lack of stats/metrics that have been talked about here make the point better than I can. I'm interested in science topics, would like to make a contribution where I can, but after a few weeks don't have much interest in continuing. I expected that I would come to learn more about the project.
I have read about the project -- impressive. I just don't know what good my current PC is doing. For example, the instruments are dathering data. The data must be processed. This takes lots of CPU cycles. So... are we close to keeping up with the data? Falling far behind? Catching up?
Hello all,
This project is very impressive...I do expect to do a great deal of valuation and revaluation computations...cause if and when the gravity wave is detected, I don't want any error's in the data that will call into quastion the results.
This is similar to the Manhattan Project, where only the best of the best where called in this time we(me) the common people get to take part.
I'm sure the people on this project could post their powerpoint on the web-site, but I think it will be well over my head. If it is data you need I'd try "http://www.ligo.org/", there is alot of stuff there.
We do appreciate that people want to see more of what is going on and are making efforts to provide more information, graphical and otherwise.
The "End of May" report card that Bruce Allen mentioned was finished at the end of June. It is now under review by the LIGO Science Collaboration.
Basically, the policy on the data has been that it represents the work of all four hundred or so Collaboration members, many of whom have sweated all day every day for a decade or more to get it to this point; and that no results using the data be made public without Collaboration approval. It would be very embarrassing if one person put out a wrong result with all our names attached to it. This doesn't mean that four hundred people have to read everything and sign off on it, which would take forever, but there are review committees that scrutinize things (down to picking the code apart) before saying OK and that takes time.
Hopefully the review is moving along and we will be able to show you all the Report Card soon. It may take a little while longer, though, because this is the first time they have contemplated "publishing" partial results in such form and they may have to sort out some broader issues. The previous results have all been technical articles and talks. As Greg pointed out, some are available at the LSC web site.
The bad news is that Bruce has disappeared. The good news is that he is incommunicado because last week Dave Anderson, father of BOINC, made a presentation at a conference nearby. Bruce saw the chance to grab him, and they are basically chained together for a couple of weeks sorting out some long-standing BOINC issues.
Here's a little report on one of the things I'm doing, which affects how the data is handled. The searches so far have only done 10 hours at a time. (There was a total of 600 hours, but divided into 60 10-hour chunks.) The sensitivity of the search is greater with longer data chunks, so we want to do that. The price is greater computational cost, of course. We think we can go up to about 50 hours in the next search with some more development work.
But also the way we divide up the parameter space becomes more of an issue (and it relates back to the cost). As the integration time (the length of a data chunk) grows beyond a day, the way we pick sky positions becomes trickier. At 10 hours, which is a fraction of a day, there isn't really enough time to see the sinusoidal Doppler shifts and their sky dependence, so we just put a grid of 30,000 equally spaced points on the sky. It's a bit of overkill in some cases, but not too much, and it's easy to implement. All the work units are the same size, for instance, though some can take a little longer if they are contaminated by excessive instrument noise.
At 50 hours, equal spacing on the sky doesn't work. There are now a couple of cycles of the varying Doppler shift due to the Earth's rotation, and some parts of the sky need to be more densely populated than others. And 50 hours is long enough that we need to allow for possible intrinsic spindown of the source (that is, the frequency drifting down with time). So for each work unit (frequency band), for each point on the sky, we also in general need to search over different values of the spindown.
I've been working on the spacing, and also on chopping up the sky. Right now each work unit does the whole sky for a 0.1Hz frequency band. In a 50 hour search, the number of points needed to cover the sky is much more at high frequencies, so each work unit has to do part of the sky if you want them all to be about the same size.
And it turns out the work units can't be made precisely the same size. There will be a range of sizes, which we're trying to pin down. Our work units are already pretty large for machines that are old or not donating much time to this project, so some people will be happy to see some shorter work units. But in contrast to other projects, we only have one server that does everything, and too many short work units will result in crashes as the server gets hit with too many requests for new work. (That's the main reason why the current work units are large and the deadline short.)
RE: It's good to know
)
Jan,
Don't forget - These people running the project don't owe us anything. Yes, we are generously letting them use our computers to run research programs, but we can't get involved out of interest and then demand something in return! That said, it's of course appreciated when we get more details on what is going on. I just think your post, Jan, had a bit too much of a demanding tone to it.
You imply that the people at Einstein@Home have to do more to keep people running the program.....well, I say that people who were interested enough to sign up for the project and download BOINC in the first place are probably not going to up and leave over some lack of detailed status reports.....
RE: You imply that the
)
Nonetheless, I find my interest is reinvigorated whenever a project scientist responds to a question or gives a brief update. For example, for a well communicated and scientifically significant project such as Folding@home, I stayed with for over three years.
The beauty of a project like Einstein is that it will produce significant scientific results, and enhance in a fundamental way our understanding of the universe. And the results will be published and available for all to see. Plus the project has its act together in terms of the solidness of its servers and its general operation.
RE: Hi folks, The numbers
)
Ben,
I feel like I should have my head examined! I never looked too closely at the server status page in terms of doing the math to see how the project was progressing. I guess I just assumed that the listed database was only a partial database, because E@h wanted to limit the number of database workunits at any given time.
But I guess I made a wrong assumption!
And it looks like with only about 14,725 Unsent results, this S3 phase of the project is about to come to an end?
Perhaps a sentence or two on the Server Status page explaining what the data means and how one can calculate some basic statistics would be useful.
Thanks for your wonderful posts! :)
RE: RE: Hi folks, The
)
Well - I've seen these numbers go up and down last few weeks.
Greetings, Mr Ragnar Schroder
RE: Jan, Don't forget -
)
I could pull plug on this project just to prove you wrong, but I won't ;-)
I guess my post was a little whiny and demanding. I can assure you that was not my intention. I can appreciate that project has limited ressources and have to prioritize their efforts.
I wanted to ensure project team understood there is a slight difference between runtime statistics and what I proposed. I guess the "keeping people interested"-part was speculative and not entirely fair way emphasize my points.
Regards
Jan H. Hansen
RE: RE: Jan, Don't
)
I, for one, signed up enthusiatically. Then I got worried when I noticed how hot my laptop got when I generously left in on 24/7 as a contribution to E@H. Now I only leave it on when I am doing something for me (letting E@H use 98% or my cpu.) So although I volunteer for free, I _do_ feel they owe me something in return for the worry I have, if not for the actual risk, of frying my computer before its time. Something more immediate than the abstract knowledge that I am helping science.
There is an old saying that "you get what you pay for." Well, the E@H folks have a very high rate of return (50,472 active users as of this moment: 50K computers -- probably $50,000,000 worth!) for very little cost. What an amazing benefit/cost ratio!! Now if they invested a little more by providing an occasional update, or perhaps even just a jazzier display on the server status page, and maintained only a fraction of their present cost/benefit ratio, then they could get a greater number of active users. Where else can one get so much for so little. This, of course, assumes they want more acive users.
If a person's behavior gets no positive reinforcement, the behavior tends to be extinguished. I used to be at SETI, where there was a nice little display showing what _my_ computer was doing. I could even zone out looking at it, hoping I would be watching when something significant happened. This was all very irrational, since I wouldn't have the faintest idea what to look for. It was a little like playing the lottery for a 1:200,000,000 chance. But it also fed my ego, for this was _my_ work (even though I had nothing at all to do with it -- it was my computer's work product). There was a little add-on that graphically showed me all of and the best of the various foobars I had found. Even though I did not really know what the foobars were, it satisfied my need to feel that I actually accomplished something. The only thing similar in E@H is the screen saver, and while it may be showing my data, there is something monotonous and impersonal about it. It would be nicer to show, for example, what frequency bank I was processing, or some pattern related to the output. Even though I might not know that it meant, I could say, "aha, I got a really big squiggle there in the left hand side this time", and I could dream that I had been a part of finding a real wave.
Please don't forget the human side of this. We are, even the mathematicians, full of irrational human tendencies. You may get more computer time donated if you package the product a little nicer, and pay attention to these irrational qualities that motivate us.
Thanks,
Tom
RE: ... There is an old
)
Well said, and I agree. I had 2 PCs running E@H. One in the attic that I set up and got running again just for this project. I recently had some in-house LAN problem, took the old PC down "temporarily," and decided not to start it up again.
Why? First of all, at $0.14/kwhr and 130 watts, it costs about $13/month to run the it. Second, I don't really know what "progress is being made." At this point, it's alll very abstract to me -- the lack of stats/metrics that have been talked about here make the point better than I can. I'm interested in science topics, would like to make a contribution where I can, but after a few weeks don't have much interest in continuing. I expected that I would come to learn more about the project.
I have read about the project -- impressive. I just don't know what good my current PC is doing. For example, the instruments are dathering data. The data must be processed. This takes lots of CPU cycles. So... are we close to keeping up with the data? Falling far behind? Catching up?
Hello all, This project is
)
Hello all,
This project is very impressive...I do expect to do a great deal of valuation and revaluation computations...cause if and when the gravity wave is detected, I don't want any error's in the data that will call into quastion the results.
This is similar to the Manhattan Project, where only the best of the best where called in this time we(me) the common people get to take part.
I'm sure the people on this project could post their powerpoint on the web-site, but I think it will be well over my head. If it is data you need I'd try "http://www.ligo.org/", there is alot of stuff there.
Greg
Hi folks, We do appreciate
)
Hi folks,
We do appreciate that people want to see more of what is going on and are making efforts to provide more information, graphical and otherwise.
The "End of May" report card that Bruce Allen mentioned was finished at the end of June. It is now under review by the LIGO Science Collaboration.
Basically, the policy on the data has been that it represents the work of all four hundred or so Collaboration members, many of whom have sweated all day every day for a decade or more to get it to this point; and that no results using the data be made public without Collaboration approval. It would be very embarrassing if one person put out a wrong result with all our names attached to it. This doesn't mean that four hundred people have to read everything and sign off on it, which would take forever, but there are review committees that scrutinize things (down to picking the code apart) before saying OK and that takes time.
Hopefully the review is moving along and we will be able to show you all the Report Card soon. It may take a little while longer, though, because this is the first time they have contemplated "publishing" partial results in such form and they may have to sort out some broader issues. The previous results have all been technical articles and talks. As Greg pointed out, some are available at the LSC web site.
The bad news is that Bruce has disappeared. The good news is that he is incommunicado because last week Dave Anderson, father of BOINC, made a presentation at a conference nearby. Bruce saw the chance to grab him, and they are basically chained together for a couple of weeks sorting out some long-standing BOINC issues.
That's the current news.
Hope this helps,
Ben
Oh, and for the
)
Oh, and for the future...
Here's a little report on one of the things I'm doing, which affects how the data is handled. The searches so far have only done 10 hours at a time. (There was a total of 600 hours, but divided into 60 10-hour chunks.) The sensitivity of the search is greater with longer data chunks, so we want to do that. The price is greater computational cost, of course. We think we can go up to about 50 hours in the next search with some more development work.
But also the way we divide up the parameter space becomes more of an issue (and it relates back to the cost). As the integration time (the length of a data chunk) grows beyond a day, the way we pick sky positions becomes trickier. At 10 hours, which is a fraction of a day, there isn't really enough time to see the sinusoidal Doppler shifts and their sky dependence, so we just put a grid of 30,000 equally spaced points on the sky. It's a bit of overkill in some cases, but not too much, and it's easy to implement. All the work units are the same size, for instance, though some can take a little longer if they are contaminated by excessive instrument noise.
At 50 hours, equal spacing on the sky doesn't work. There are now a couple of cycles of the varying Doppler shift due to the Earth's rotation, and some parts of the sky need to be more densely populated than others. And 50 hours is long enough that we need to allow for possible intrinsic spindown of the source (that is, the frequency drifting down with time). So for each work unit (frequency band), for each point on the sky, we also in general need to search over different values of the spindown.
I've been working on the spacing, and also on chopping up the sky. Right now each work unit does the whole sky for a 0.1Hz frequency band. In a 50 hour search, the number of points needed to cover the sky is much more at high frequencies, so each work unit has to do part of the sky if you want them all to be about the same size.
And it turns out the work units can't be made precisely the same size. There will be a range of sizes, which we're trying to pin down. Our work units are already pretty large for machines that are old or not donating much time to this project, so some people will be happy to see some shorter work units. But in contrast to other projects, we only have one server that does everything, and too many short work units will result in crashes as the server gets hit with too many requests for new work. (That's the main reason why the current work units are large and the deadline short.)
Hope this helps,
Ben