"Project is down" for 19 hours now

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4266
Credit: 244926081
RAC: 16571

Status: Good: Database is

Status:

Good:
Database is running more smoothly than ever, daemons (transitioner, assimilator, validator) are working through their backlogs, scheduler seems to send out work.

Bad:
All related to ABP1 is currently offline, because for some reason the connection to Hannover isn't stable. For some other reason the S5R5 workunit generator is running too slow to keep up with work requests, so the project might run out of work some time over the weekend, unless someone fixed these issues.

But after some 12 hours of hard work Oliver and I are too tired to take much care of this, we'll possibly do more damage than good. Need to get some sleep.

BM

BM

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2139
Credit: 2752801655
RAC: 1436958

I think that's a very wise,

I think that's a very wise, and well-earned, decision. If there's not a ready supply of ready-prepared work, so what? It can wait until next week, when hopefully the solution will seem more obvious.

Elphidieus
Elphidieus
Joined: 20 Feb 05
Posts: 245
Credit: 20603702
RAC: 0

Welp, my towers are going

Welp, my towers are going back to work fully-loaded...

My thanks to BM and the Einstein team for getting the project up and running again...

And not forgetting those who'd keep the forum alive... :P

Holmis
Joined: 4 Jan 05
Posts: 1118
Credit: 1055935564
RAC: 0

RE: But after some 12

Message 92523 in response to message 92520

Quote:

But after some 12 hours of hard work Oliver and I are too tired to take much care of this, we'll possibly do more damage than good. Need to get some sleep.

BM

I only want to say thank you for your hard work and I'm happy that were back in business again!

/Holmis

prairie69
prairie69
Joined: 21 Oct 07
Posts: 6
Credit: 538161
RAC: 0

RE: RE: But after some

Message 92524 in response to message 92523

Quote:
Quote:

But after some 12 hours of hard work Oliver and I are too tired to take much care of this, we'll possibly do more damage than good. Need to get some sleep.

BM

I only want to say thank you for your hard work and I'm happy that were back in business again!

/Holmis

Ditto. I can only vaguely imagine the hard work you've been doing. We're all thankful the project is up and running again.

prairie69

MarkJ
MarkJ
Joined: 28 Feb 08
Posts: 437
Credit: 137327847
RAC: 18476

RE: But after some 12 hours

Message 92525 in response to message 92520

Quote:

But after some 12 hours of hard work Oliver and I are too tired to take much care of this, we'll possibly do more damage than good. Need to get some sleep.

BM

Thank you (and oliver) for you hard work to get the project back up. Work is flowing again. Once again thanks guys.

Stranger7777
Stranger7777
Joined: 17 Mar 05
Posts: 436
Credit: 417508669
RAC: 33751

I think any growing project

I think any growing project sooner or later will faces with such a problem. The problem of growing table size is so common that it is time to think about a new ways to manage data. Purging and indexing the table more and more will not get a fix for it. It'll be better to decide how to divide big tables into subarrays which size will be sufficiently smaller to not reach the access speed limit in the near future. One of the most popular program in Russia is 1C:Accounting 1C:Accounting because of lot of accounting laws and enourmous number of taxes. This program already prove itself as an excellent database manager - it can struggle agains gigabyte-size databases without any huge requests to a hardware. Moreover some old versions did work with *.dbf tables and it was not terrible - it works these days on many millions computers allover the Russia, Ukraine, Belarus etc for years and it is not slowing down. Of course today they build already new versions that works with MySQL or MS-SQL, but this is the fact - this version is still works and is supported by the developers yet.
So, this time we faced the problem that says us: you may rely on sql technologies, but you surely must remember how did you worked with this data about 10 years ago - those technologies are not dead yet.
Guess, this should help a bit :)

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4266
Credit: 244926081
RAC: 16571

Update: * We found a

Update:

* We found a workaround for the problems with the S5R5 WUG (WU Generator), so we probably won't run out of work in the near future.

* We also found a workaround to run the ABP1 validator and assimilator, the WUG is still causing some headache.

* We needed a couple of workarounds to bring the project back online, which will probably require a scheduled downtime for a few hours some time next month to remove them. But for now we're getting back on track.

BM

BM

Vagn
Vagn
Joined: 3 May 08
Posts: 2
Credit: 586927540
RAC: 452500

RE: Update: * We needed a

Message 92528 in response to message 92527

Quote:

Update:

* We needed a couple of workarounds to bring the project back online, which will probably require a scheduled downtime for a few hours some time next month to remove them. But for now we're getting back on track.

BM


Does that remaining work include communications / connection to say : boincstats.com and statsnstones.tswb.org ?
I've noticed that they haven't been updating Einstein data for a while now. So when can we expect that to happen ?
Or am I wrong and it isn't an Einstein issue but something related to mentioned statistics sites ?

regards
Vagn

mikey
mikey
Joined: 22 Jan 05
Posts: 11889
Credit: 1828127138
RAC: 206028

RE: RE: Update: * We

Message 92529 in response to message 92528

Quote:
Quote:

Update:

* We needed a couple of workarounds to bring the project back online, which will probably require a scheduled downtime for a few hours some time next month to remove them. But for now we're getting back on track.

BM


Does that remaining work include communications / connection to say : boincstats.com and statsnstones.tswb.org ?
I've noticed that they haven't been updating Einstein data for a while now. So when can we expect that to happen ?
Or am I wrong and it isn't an Einstein issue but something related to mentioned statistics sites ?
regards Vagn

Most projects provide a 'feed' of the stats, if one independent site gets them then they all can. Some independent sites are not real good about tracking when they do not get the stats and it can take a few days to figure out where they went if the project moves their 'feed'.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.