"Project is down" for 19 hours now

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: RE: One other item

Message 92500 in response to message 92495

Quote:
Quote:

One other item needs to be considered when the servers are put back on line, is the fact that some tasks that would have been reported in time are now past their deadline.

So I hope the Validators and any other process that re-issues because of missed deadlines are delayed, for at least 24 hours.

If a unit gets re-issued because it wasn't returned before the deadline, and the original person returns it before the 'new' person, the original person gets the credit, as long as it is a valid crunch. I set my pc's to no new work, partly so I wouldn't be involved in all the day to day worries right now, but also to avoid this very problem. I do not want to get your past deadline unit, you return it and then I don't get credit for crunching it! After the Project is back up and running fine, I will come back. There are plenty of other projects out there that have short units, and need our help too.

Except, that's not what would happen if your host got the reissue.

The worst case for the host which draws a reissued task is it might turn out to be superfluous, but would still get credit.

If the originals report first and you're lucky, your host wouldn't have started it yet and it would get 221'ed on the next scheduler request.

Only hosts which are over the deadline are at risk of getting burned if a new wingman reports back before they can get through, assuming the WU already has one task returned. You can extrapolate that to the case where both the original replications are over the deadline as well.

Alinator

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: RE: RE: RE: You

Message 92501 in response to message 92498

Quote:
Quote:
Quote:
Quote:

You're forgetting the environment that BOINC is targeted to run in: science projects on a shoe-string budget.

And although Mainframes etc. don't need to have RAM > DB size because they generally run "proper" databases such as DB2 or Oracle, the "proper" databases cost $$$ more than MySQL.

But I'm pretty sure a small-ish IBM i5 could replace a large-RAM x86 server well enough and cheaply enough (you get DB2 for free with an i5).

Well, I guess BOINC server side code only works with MySQL (or does it have adaptors for other DBs???), but hey, as MySQL was bought by Sun and Sun was now bought by Oracle ... who knows where MySQL will go in the end :-)

CU
Bikeman


Some of the databases on Seti are Informix Matt Lebofsky post, so guess any SQL DB could be used.

Agreed, there is nothing that says you have to use MySQL for the BOINC database.

However, even SAH uses MySQL for it. The Master Science Database runs on Informix. IIRC, it was donated to the project back in the SERENDIP (or maybe Classic) days and has just sort of been handed down the line as the project has evolved. ;-)

Alinator

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2139
Credit: 2753028342
RAC: 1371490

If reissues are sent out

If reissues are sent out prematurely, without allowance for the 24-hour reporting backoffs in recent clients, then both bandwidth and database table size grow unnecessarily, and we're back with the problem we started with.

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: If reissues are sent

Message 92503 in response to message 92502

Quote:
If reissues are sent out prematurely, without allowance for the 24-hour reporting backoffs in recent clients, then both bandwidth and database table size grow unnecessarily, and we're back with the problem we started with.

That's true enough, however in this case I think it has more to do with the generous amount of time EAH lets the completed WUs hang around before they go poof. ;-)

In any event, I wouldn't want to see them go to anything less than the minimum standard interval of 1 day permanently. Certainly not Insta-Purge like they did on MW a while back. Now that was a real annoyance for us datahounds! :-D

Alinator

BarryAZ
BarryAZ
Joined: 8 May 05
Posts: 190
Credit: 320773870
RAC: 11636

For me, when Einstein does

Message 92504 in response to message 92503

For me, when Einstein does come back on line (24 hours from whenever you ask) it will be a restart for me, I suspended processing on Einstein several days ago and then aborted much of the unstarted queued work as it came up against due date. So when it does start up, I'll be reporting killed off work units and looking to download new work. I'll probably back off on resource share to start so that I won't be looking for a lot of downloaded work during the first post crash week.

Jord
Joined: 26 Jan 05
Posts: 2952
Credit: 5779100
RAC: 0

RE: Agreed, there is

Message 92505 in response to message 92501

Quote:
Agreed, there is nothing that says you have to use MySQL for the BOINC database.


There is: http://boinc.berkeley.edu/trac/wiki/SoftwarePrereqsUnix and http://boinc.berkeley.edu/trac/wiki/DataBase

There is nothing that says that you have to use MySQL for the science database.

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: RE: RE: RE: Quote

Message 92506 in response to message 92501

Quote:
Quote:
Quote:
Quote:
Quote:

You're forgetting the environment that BOINC is targeted to run in: science projects on a shoe-string budget.

And although Mainframes etc. don't need to have RAM > DB size because they generally run "proper" databases such as DB2 or Oracle, the "proper" databases cost $$$ more than MySQL.

But I'm pretty sure a small-ish IBM i5 could replace a large-RAM x86 server well enough and cheaply enough (you get DB2 for free with an i5).

Well, I guess BOINC server side code only works with MySQL (or does it have adaptors for other DBs???), but hey, as MySQL was bought by Sun and Sun was now bought by Oracle ... who knows where MySQL will go in the end :-)

CU
Bikeman


Some of the databases on Seti are Informix Matt Lebofsky post, so guess any SQL DB could be used.

Agreed, there is nothing that says you have to use MySQL for the BOINC database.

However, even SAH uses MySQL for it. The Master Science Database runs on Informix. IIRC, it was donated to the project back in the SERENDIP (or maybe Classic) days and has just sort of been handed down the line as the project has evolved. ;-)

Alinator

Pardon the bad form of replying to myself. However the reason should be clear. ;-)

Copied from a PM from Der Meister:

Quote:

Hi,
in this thread you said that probably any DBMS could be used as database instead of MySQL. I would really like to add an answer to this thread but BOINC doesn't let me because of "not enough credits"... Well, I have more than 50k here but I guess the RAC is meant which is indeed quite low for me at the moment.

Anyway, back to topic:
The BOINC server can only use MySQL and nothing else. Each server component (scheduler, backend daemons and even the webpages) are strongly coupled with the MySQL interface. There is no real abstraction layer (although some class names might suggest this) that would allow the server admin to use the BOINC server code with a different database system. This is one of the biggest design deficies of the BOINC server and I guess this will never get fixed. This leaves the project (especially the bigger ones) with lots of database related issues. And as Sun was bought by Oracle I'd even say MySQL's future is quite unclear. I don't think that Oracle will improve MySQL as it might become a threat to their own database system then.

It would be nice if you could add this information to the thread mentioned above. :)

Thanks and greetings,
Der Meister

Will do about transcribing your reply to the NC forum.

You're right of course about the standard package needing MySQL. I guess I should have worded the post to say that you can use any other one you want if you take the time and effort to port the rest of the package to the target DB platform.

Regards,

Alinator

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686209377
RAC: 552085

Note the Update on the News

Note the Update on the News section of the Homepage

Quote:

Apr 23, 2009
The database is still purging, at a rate of about 400,000 workunits per day. We expect that this process will be completed by about 06:00 UTC tomorrow. At that time, we will do some additional database maintenance (table optimization) and then restart the project. So in total about another 24 hours will still be needed to bring everything back to normal. Thanks again for your patience during this unfortunate and long outage.


CU
Bikeman

Stan Pope
Stan Pope
Joined: 22 Dec 05
Posts: 80
Credit: 426811575
RAC: 0

I really appreciate the more

I really appreciate the more frequent "situation updates" provided during the last couple of days. Few things irk me more than seeing the project down or dying and no indication that anyone of responsibility even knows there is a problem.

Stan

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9352143
RAC: 0

RE: Note the Update on the

Message 92509 in response to message 92507

Quote:

Note the Update on the News section of the Homepage

Quote:

Apr 23, 2009
The database is still purging, at a rate of about 400,000 workunits per day. We expect that this process will be completed by about 06:00 UTC tomorrow. At that time, we will do some additional database maintenance (table optimization) and then restart the project. So in total about another 24 hours will still be needed to bring everything back to normal. Thanks again for your patience during this unfortunate and long outage.

CU
Bikeman

LOL...

Yep, I saw that. Then the thought occurred to me, isn't that only about ten times faster than we plow through the work on our end? ;-)

Alinator

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.