It would seem to me a line count would be less taxing on resources than listing them out. But.. I am not a programmer.
The current "pendings" page is a nightmare on its own, both for the user and the system. I would rather spend time on fixing / improving this than extending the "tasks" pages. Don't know when I find the time for either, though.
Must of us a are returning to our homeland soon, but before we do that, i wish to thanks all for the good time we pass in this 3 weeks. Was a pleasure to meet so many new friends.
I´m realy hope you liked our contribution to the project in this time (about 350PFlops) and maybe in the future we could do more. If the door´s remains opens, why not? We know we could do better, but take some time to learn who to tunned our hungry hosts to crunch E@H a little better, and still a hard and long task to realy learn who to do that with eficience, to much to learn in so little time. We left a large number of pendigs, so that will be add a litlle more Tflops to the pot. Hope at least 1 discovery could be made with all that data.
Keep focus on the project, you all have a hard and long task to do, but i´m sure there is a lot of new discoveries waiting.
To the project admins, an all who give us a suport, thanks for all and hope we could see you all again.
We don´t close our participation in the project, but for now our main SETI team need us, but we will return. The last weeks was to good to forget.
It would seem to me a line count would be less taxing on resources than listing them out. But.. I am not a programmer.
The current "pendings" page is a nightmare on its own, both for the user and the system. I would rather spend time on fixing / improving this than extending the "tasks" pages. Don't know when I find the time for either, though.
BM
HOW does the current page work? Does it generate the page every time a user asks for it? If so that WOULD be a nightmare for an already busy server! ASSUMING that IS how it works maybe a second computer that you can periodically send that info to say once an hour, and then have that pc generate the report from there which would lessen the main Servers workload a bit.
I just thought a line count (which with some exception IS the information most of us look for on those pages.. MOST of the time..) would be much less taxing to generate than the lists themselves. And I would concur an occasional output of that information would in most cases be adequate. I also am sympathetic to programmers nightmares of code. Once upon a time I found myself thinking in Korn Shell and knew at that point I needed to change career directions.
I just thought a line count (which with some exception IS the information most of us look for on those pages.. MOST of the time..) would be much less taxing to generate than the lists themselves. And I would concur an occasional output of that information would in most cases be adequate. I also am sympathetic to programmers nightmares of code. Once upon a time I found myself thinking in Korn Shell and knew at that point I needed to change career directions.
The beauty of databases is that the code is very easy - in this case, just one line in pending.php:
$results = BoincResult::enum("userid=$user->id AND (validate_state=0 OR validate_state=4) AND claimed_credit > 0");
But that one line can tie up a server for several seconds while it searches through three million tasks in the database and works out which belong to you (and are pending).
But that one line can tie up a server for several seconds while it searches through three million tasks in the database and works out which belong to you (and are pending).
Often database resource consumption gets down both to what keys are available in the chosen database structure, and just how the query is written.
If using a key selects only a small number of lines, even slightly wasteful subsequent processing may not be very costly. But anything which really requires a search on every line is anathema. I'm no database expert, but was doing "big data" on multi-gigabyte manufacturing databases before that term came into vogue, and even taught a couple of sessions of SQL classes. Aside from technical detail, the two major points I tried to drum into students' heads were the adroit use of keys, and the huge benefit of server-side summarization (as in our case the pipe from the database server to our desktops had astonishingly low capacity).
We, of course, were stuck with the key provision of the database implementation we were using. I'll hazard a guess that making pending counts or queries work well at Einstein might require revision either in key provision or in the formulation of the queries.
None of this commentary is aimed at either project staff or Richard, all of whom I assume to know much more than I on these matters--but rather as an aside to my fellow users.
The beauty of databases is that the code is very easy - in this case, just one line in pending.php:
$results = BoincResult::enum("userid=$user->id AND (validate_state=0 OR validate_state=4) AND claimed_credit > 0");
But that one line can tie up a server for several seconds while it searches through three million tasks in the database and works out which belong to you (and are pending).
One problem is that BOINC's DB interface (here BoincResult) partially ruins one possible benefit of having a DB by transferring every field of the result record (think of blobs like xml_doc, stderr_out etc.) to PHP memory even if only a few simple fields are ever shown on the web page.
Doing this without any limitation to the number of records to fetch kills the DB connection just by I/O from the DB if the number of records is large enough. But fixing the root cause of that would require to change the whole BOINC PHP DB interface, not just this page.
Must of us a are returning to our homeland soon, but before we do that, i wish to thanks all for the good time we pass in this 3 weeks. Was a pleasure to meet so many new friends.
I´m realy hope you liked our contribution to the project in this time (about 350PFlops) and maybe in the future we could do more. If the door´s remains opens, why not? We know we could do better, but take some time to learn who to tunned our hungry hosts to crunch E@H a little better, and still a hard and long task to realy learn who to do that with eficience, to much to learn in so little time. We left a large number of pendigs, so that will be add a litlle more Tflops to the pot. Hope at least 1 discovery could be made with all that data.
Keep focus on the project, you all have a hard and long task to do, but i´m sure there is a lot of new discoveries waiting.
To the project admins, an all who give us a suport, thanks for all and hope we could see you all again.
We don´t close our participation in the project, but for now our main SETI team need us, but we will return. The last weeks was to good to forget.
Thanks very much for your substantial contribution, and of course you are welcome to return any time. Needless to say, participating in the forum here is independent from contributing to the project (unless your RAC decays to 0 , for spam protection), so feel free to check out the forum and stay connected.
The beauty of databases is that the code is very easy - in this case, just one line in pending.php:
$results = BoincResult::enum("userid=$user->id AND (validate_state=0 OR validate_state=4) AND claimed_credit > 0");
But that one line can tie up a server for several seconds while it searches through three million tasks in the database and works out which belong to you (and are pending).
One problem is that BOINC's DB interface (here BoincResult) partially ruins one possible benefit of having a DB by transferring every field of the result record (think of blobs like xml_doc, stderr_out etc.) to PHP memory even if only a few simple fields are ever shown on the web page.
Doing this without any limitation to the number of records to fetch kills the DB connection just by I/O from the DB if the number of records is large enough. But fixing the root cause of that would require to change the whole BOINC PHP DB interface, not just this page.
BM
Yes, I was surprised that the line I was reading didn't look like
SELECT result.id, result.workunitid, result.hostid, result.claimed_credit FROM result WHERE ....
RE: It would seem to me a
)
The current "pendings" page is a nightmare on its own, both for the user and the system. I would rather spend time on fixing / improving this than extending the "tasks" pages. Don't know when I find the time for either, though.
BM
BM
Must of us a are returning to
)
Must of us a are returning to our homeland soon, but before we do that, i wish to thanks all for the good time we pass in this 3 weeks. Was a pleasure to meet so many new friends.
I´m realy hope you liked our contribution to the project in this time (about 350PFlops) and maybe in the future we could do more. If the door´s remains opens, why not? We know we could do better, but take some time to learn who to tunned our hungry hosts to crunch E@H a little better, and still a hard and long task to realy learn who to do that with eficience, to much to learn in so little time. We left a large number of pendigs, so that will be add a litlle more Tflops to the pot. Hope at least 1 discovery could be made with all that data.
Keep focus on the project, you all have a hard and long task to do, but i´m sure there is a lot of new discoveries waiting.
To the project admins, an all who give us a suport, thanks for all and hope we could see you all again.
We don´t close our participation in the project, but for now our main SETI team need us, but we will return. The last weeks was to good to forget.
RE: RE: It would seem to
)
HOW does the current page work? Does it generate the page every time a user asks for it? If so that WOULD be a nightmare for an already busy server! ASSUMING that IS how it works maybe a second computer that you can periodically send that info to say once an hour, and then have that pc generate the report from there which would lessen the main Servers workload a bit.
I just thought a line count
)
I just thought a line count (which with some exception IS the information most of us look for on those pages.. MOST of the time..) would be much less taxing to generate than the lists themselves. And I would concur an occasional output of that information would in most cases be adequate. I also am sympathetic to programmers nightmares of code. Once upon a time I found myself thinking in Korn Shell and knew at that point I needed to change career directions.
RE: I just thought a line
)
The beauty of databases is that the code is very easy - in this case, just one line in pending.php:
$results = BoincResult::enum("userid=$user->id AND (validate_state=0 OR validate_state=4) AND claimed_credit > 0");
But that one line can tie up a server for several seconds while it searches through three million tasks in the database and works out which belong to you (and are pending).RE: But that one line can
)
Often database resource consumption gets down both to what keys are available in the chosen database structure, and just how the query is written.
If using a key selects only a small number of lines, even slightly wasteful subsequent processing may not be very costly. But anything which really requires a search on every line is anathema. I'm no database expert, but was doing "big data" on multi-gigabyte manufacturing databases before that term came into vogue, and even taught a couple of sessions of SQL classes. Aside from technical detail, the two major points I tried to drum into students' heads were the adroit use of keys, and the huge benefit of server-side summarization (as in our case the pipe from the database server to our desktops had astonishingly low capacity).
We, of course, were stuck with the key provision of the database implementation we were using. I'll hazard a guess that making pending counts or queries work well at Einstein might require revision either in key provision or in the formulation of the queries.
None of this commentary is aimed at either project staff or Richard, all of whom I assume to know much more than I on these matters--but rather as an aside to my fellow users.
RE: The beauty of databases
)
One problem is that BOINC's DB interface (here BoincResult) partially ruins one possible benefit of having a DB by transferring every field of the result record (think of blobs like xml_doc, stderr_out etc.) to PHP memory even if only a few simple fields are ever shown on the web page.
Doing this without any limitation to the number of records to fetch kills the DB connection just by I/O from the DB if the number of records is large enough. But fixing the root cause of that would require to change the whole BOINC PHP DB interface, not just this page.
BM
BM
RE: Must of us a are
)
Thanks very much for your substantial contribution, and of course you are welcome to return any time. Needless to say, participating in the forum here is independent from contributing to the project (unless your RAC decays to 0 , for spam protection), so feel free to check out the forum and stay connected.
Cheers and feline greetings.
HB
RE: RE: The beauty of
)
Yes, I was surprised that the line I was reading didn't look like
SELECT result.id, result.workunitid, result.hostid, result.claimed_credit FROM result WHERE ....
RE: Thanks very much for
)
Could speak only for me, "i will back". I realy like to participate in your project. All reamining E@H in my host will be crunched in time.
My best wishes for your kitties, leave you with this song spacialy for them.
http://www.youtube.com/watch?v=_9uCP7jIa1I