Overcompensation

Gerry Rough
Gerry Rough
Joined: 1 Mar 05
Posts: 102
Credit: 1,847,066
RAC: 0
Topic 194283

I have for several years wondered why BOINC overcompensates when a project goes offline, even it seems, when the offline is only for a few hours. If one project goes offline, other projects automatically download new WUs to keep the client busy. But when they do so, they often download far too many WUs to simply make up for a few hours offline. Sure enough, it sends BOINC into EDF mode for a few hours to a day or longer to catch up.

I should think there is a simple fix. For every project there are resource preferences, including the project that is temporarily offline, whether hours or even for months. What about the idea that any project that queries the server for more work should be limited to the resource share given by the user plus, say, 20% of that resource share?

So for example, no matter what happens, even if all other projectrs went offline simultaneously, an obvously worst case scenario, any given project would only download the requirement of the resource preferences plus 20% of that resource share requirement. That way BOINC would not end up in WU la-la land, end up in EDF mode, and BOINC would only download enough to keep busy rather than fill the cache with too much work temporarily. And, when the offline project starts issuing new work again, there would be little if any catching up to do. The other projects would simply not download any new work until they finished with the ones already in the client's cache.


(Click for detailed stats)

Paul D. Buck
Paul D. Buck
Joined: 17 Jan 05
Posts: 754
Credit: 5,385,205
RAC: 0

Overcompensation

Quote:

I have for several years wondered why BOINC overcompensates when a project goes offline, even it seems, when the offline is only for a few hours. If one project goes offline, other projects automatically download new WUs to keep the client busy. But when they do so, they often download far too many WUs to simply make up for a few hours offline. Sure enough, it sends BOINC into EDF mode for a few hours to a day or longer to catch up.

I should think there is a simple fix. For every project there are resource preferences, including the project that is temporarily offline, whether hours or even for months. What about the idea that any project that queries the server for more work should be limited to the resource share given by the user plus, say, 20% of that resource share?

So for example, no matter what happens, even if all other projectrs went offline simultaneously, an obvously worst case scenario, any given project would only download the requirement of the resource preferences plus 20% of that resource share requirement. That way BOINC would not end up in WU la-la land, end up in EDF mode, and BOINC would only download enough to keep busy rather than fill the cache with too much work temporarily. And, when the offline project starts issuing new work again, there would be little if any catching up to do. The other projects would simply not download any new work until they finished with the ones already in the client's cache.


Heresy, Heresy I say sir ...

Were you not one of those telling me how perfectly BOINC works if you just leave it alone?

If you are not careful you will have lots of people calling you a whiner and someone that does not understand how well BOINC works, if you just ignore all those "little" things that might indicate that it does not ...

Gerry Rough
Gerry Rough
Joined: 1 Mar 05
Posts: 102
Credit: 1,847,066
RAC: 0

RE: Heresy, Heresy I say

Message 92365 in response to message 92364

Quote:

Heresy, Heresy I say sir ...

Were you not one of those telling me how perfectly BOINC works if you just leave it alone?

If you are not careful you will have lots of people calling you a whiner and someone that does not understand how well BOINC works, if you just ignore all those "little" things that might indicate that it does not ...

Somehow me thinks there is a compliment in there somewhere. Must have been body language!

Generally speaking, you are correct. I have mentioned elsewhere to leave it alone to let BOINC work out the problems. But that only goes so far. If BOINC has a built in problem, that is quite another matter.

At least I know I'm in good company. :0)

It doesn't bother me at all that BOINC has its kinks that need to get worked out, but it does make me wonder on occasion that a number of chronic "smaller" issues continue. BOINC, as has been stated on these boards many-a-time, is quite mature; we've come a long way. It surprises me that issues like this still continue with such a mature program.


(Click for detailed stats)

Winterknight
Winterknight
Joined: 4 Jun 05
Posts: 577
Credit: 215,961,094
RAC: 108,518

I asked JM7 about this

I asked JM7 about this problem. He said that it had been changed, at the request of users, so that your work cache is kept full.

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9,352,143
RAC: 0

Well, the reason for the

Well, the reason for the change from what it was in 4x was that if you had a truly intermittent network connection (dialup) and you only could pull one task at a time from debt ineligible projects (and then only if you were out of work for eligible ones), it was likely that the host would run out of work entirely before the next connection opportunity.

In this case I had to agree with the logic. My beef was then (and still is), the new policy is just fine and dandy for taking care of that problem, but if you have an always available connection isn't that just asking to cause share breaks for no good reason?

I mean so what if when you are carrying a max cache the CC runs it down a little when one of your debt eligible projects goes offline temporarily. Isn't that exactly what the cache is for?

It is completely retarded to take on a ton of work for an overworked project on an always connected host just to make sure the cache stays full to the max at all times. The odds are you would get the work the host really wanted originally in a few minutes to hours more or less (overlooking the somewhat unique situation at MW currently).

The argument that giving an option to explicitly say whether the host was truly network connection limited or not was 'too much complexity' was then and still is a load of codswallop. Especially since AFAIK BOINC has never been able to determine anything about the network state with any kind of accuracy, and never has been able to figure out what to do with a modem when it's done using it (at least on Windows through 5.10.13).

Alinator

Winterknight
Winterknight
Joined: 4 Jun 05
Posts: 577
Credit: 215,961,094
RAC: 108,518

Of course the real problem,

Of course the real problem, is when the project comes back on line.
BOINC has filled your cache to overflowing, because of the ask for one second of work - get many hours, and then it decides to fill the errant projects cache.

My cache is now ~30% larger than requested, because I just got a load of work, with a new application Hierarchical S5 all-sky GW search #5 3.04, thats for Windows.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,356
Credit: 183,018,300
RAC: 643,716

RE: BOINC, as has been

Message 92369 in response to message 92365

Quote:
BOINC, as has been stated on these boards many-a-time, is quite mature; we've come a long way. It surprises me that issues like this still continue with such a mature program.


Heck it surprises me that it works at all! Mind you I'm easily boggled ..... :-)
It's such an ambitious framework, with disparate inputs and assumptions, and a plethora of unique instances/installs out there .... and it hasn't fallen over dead, yet, unlike most committee camels. :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9,352,143
RAC: 0

RE: Of course the real

Message 92370 in response to message 92368

Quote:

Of course the real problem, is when the project comes back on line.
BOINC has filled your cache to overflowing, because of the ask for one second of work - get many hours, and then it decides to fill the errant projects cache.

My cache is now ~30% larger than requested, because I just got a load of work, with a new application Hierarchical S5 all-sky GW search #5 3.04, thats for Windows.

UGGGHHH!!!

LOL...

I thought they had gotten that little OOPs out of the backend source code already! :-)

Funny how those little buggers resist being stamped out once and for all. ;-)

Might have something to do with the fine state of server source code documentation Paul was commenting on elsewhere. :-D

Alinator

Alinator
Alinator
Joined: 8 May 05
Posts: 927
Credit: 9,352,143
RAC: 0

RE: RE: BOINC, as has

Message 92371 in response to message 92369

Quote:
Quote:
BOINC, as has been stated on these boards many-a-time, is quite mature; we've come a long way. It surprises me that issues like this still continue with such a mature program.

Heck it surprises me that it works at all! Mind you I'm easily boggled ..... :-)
It's such an ambitious framework, with disparate inputs and assumptions, and a plethora of unique instances/installs out there .... and it hasn't fallen over dead, yet, unlike most committee camels. :-)

Cheers, Mike.

Hmmmm...

Well, I suppose if you take a critical look at the overall situation with BOINC projects this year so far, you could argue a pretty convincing case that a major coronary is waiting in the wings. :-D

Of course, it could just be all due to global warming and Wall Street greed. ;-)

Alinator

Gerry Rough
Gerry Rough
Joined: 1 Mar 05
Posts: 102
Credit: 1,847,066
RAC: 0

RE: It is completely

Message 92372 in response to message 92367

Quote:

It is completely retarded to take on a ton of work for an overworked project on an always connected host just to make sure the cache stays full to the max at all times. The odds are you would get the work the host really wanted originally in a few minutes to hours more or less (overlooking the somewhat unique situation at MW currently).

The argument that giving an option to explicitly say whether the host was truly network connection limited or not was 'too much complexity' was then and still is a load of codswallop. Especially since AFAIK BOINC has never been able to determine anything about the network state with any kind of accuracy, and never has been able to figure out what to do with a modem when it's done using it (at least on Windows through 5.10.13).

Alinator

I wuv you man!! 8-) [sniffle]

Now that we've all vented according to our BOINC venting preferences, what about the solution? Is the idea of preferences plus a predetermined % (20% in my example) a workable idea? What would it take to solve the problem? From where I'm sitting, limiting the number of seconds to be downloaded beyond the preferences would be an easy fix. Pretty much no fuss, no muss. I certainly don't see a downside.

You bring up a valid point about the network state, but BOINC does this all of the time doesn't it? I've seen many times in the messages tab that BOINC checks the internet connection to determine if the project is offline, then tells us in the next line of message: "internet connection is okay, project likely down for maintenence" or thereabouts. If indeed BOINC can do this, all that is needed then is to tell the client that, 'Gerry has a current connection, continue to download work.' You don't need to know if the connection is offline 12 hours per day; you have the connection now, so maintain the cache according to preferences now. Am I missing something here? As long as the client maintains the cache according to preferences (if the network connection is offline 12 hours per day, this assumption would be included in the user preferences anyway), I don't see where the client can get all tangled into a mess.


(Click for detailed stats)

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6,356
Credit: 183,018,300
RAC: 643,716

RE: Hmmmm... Well, I

Message 92373 in response to message 92371

Quote:

Hmmmm...

Well, I suppose if you take a critical look at the overall situation with BOINC projects this year so far, you could argue a pretty convincing case that a major coronary is waiting in the wings. :-D

Of course, it could just be all due to global warming and Wall Street greed. ;-)


I just have this vision of a lot of people with painted faces and funny red noses running about putting up a big circus tent ( please, no offense intended ). Someone tightens a rope here, it leans a bit there, the wind blows, a stuntman sans parachute falls in the middle, a door flap catches fire, the elephant gets upset in untidy ways, audience claps, a small handkerchief with 'BANG' written on it pops out of the gun .....

No, I haven't been drinking. Not today anyway. It's just that the O in BOINC is for Open and that always produces momentum for sure, but co-ordinated? Certainly no shortage of opinions. Made of strong stuff these devs, anyway I'll stop speaking out of turn now .... :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.