Why are deadlines so *short*?

Brian Silvers
Brian Silvers
Joined: 26 Aug 05
Posts: 772
Credit: 282700
RAC: 0

RE: Nobody should be

Message 75305 in response to message 75304

Quote:
Nobody should be expecting 24/7 operation.

Thanks for pointing that out. It was not my expectation that everyone run 24/7. I do (mostly), but my point was there to illustrate "best case scenario". Obviously if the system is off some, then that is taken into account. The work scheduler does take that sort of thing into consideration when making fetch decisions and "panic" decisions.

Quote:


So we shouldn't expect a computer to be able to crunch more than about 20hrs per week for BOINC projects total.

If the computer is attached to two projects, one of which is Einstein then a unit cannot be expected to take more than 20hrs to complete.

As observed most users don't like being in EDF mode, so it would appear that Einstein only wants the most modern computers to be attached. As even an AMD 185 hostid=887005 is taking 15+ hrs to complete a unit.

It's taking that long for that system because they're using the stock 4.02 Linux app which has some slower code in it for debug purposes. The beta Linux app, which still doesn't have SSE optimization, is taking another Opteron 185 / Linux host only 10-10.5 hours for the same type of WU. Since a 185 is typically clocked at 2.6GHz, that would put it at about the same performance as mine, which mine takes 11 on average (good ole AMD/Windows penalty showing up there, although I don't know if perhaps they have overclocked too, but it doesn't look like they have because their benchmarks are slower than mine).

That said, your quote above seems to be going in the direction of "I want to hook up my 486 and if it takes a year, I DEMAND THAT IT BE ACCEPTED!" While I can certainly agree that deadlines be loosened some due to the alpha/beta level of the application, going along with your idea of that we assume a max of 20 hours for 100% allocation and that continues to get reduced if you add more projects, then when does it stop? Must the project(s) just do coarse searches rather than finer detail searches? Should projects aiming to find cures for various diseases just say "well, a few more thousand people dying is ok because we're allowing John's Pentium 60 to finish work in 3 months instead of just saying that his computer isn't fast enough to do what we need to be doing"????

I'm fairly certain you were around for the pain and suffering of SETI 2.x to 3.x. My poor AMD K6-2 went from about 45 minutes per WU to like 6-8 hours per (memory is a bit fuzzy, I just remember it was a LARGE increase). There was much writhing and complaining, from what I remember, from people with 486 systems and first gen Pentiums.

Quote:

Therefore it is my view that deadlines are too short, especially as the applications have not been fully developed before release.

The deadlines for S5R3 are maybe 2-4 days too short. MAYBE. Even if I were to cut back to 25% uptime, at 33% allocation that would afford me 27.72 hours across 2 weeks to give to Einstein. In that amount of time, I could complete 2 results comfortably. A 3rd result wouldn't make it, EDF or not, but it would probably still get credit as it would come in only about 5-8 hours after deadline, thus making the extra result issued redundant (assuming mine was strongly similar). Cutting back further, to your 20 hour maximum weekly CPU time, my 33% allocation would still complete 1 task in time. 40 x 0.33 x 3600 = 47,520 seconds. My max runtime has been about 46,000 seconds.

If every project started having huge deadlines, what we'd see are angry participants who did the work quickly, but have gotten paired with someone who has downloaded a bunch of stuff and just quit or they are off dedicating a high percentage to some other project. The "pain" of the EDF is not anywhere near as "intolerable" to most as the "pain" of doing the work and not being rewarded in a timely manner. Look at the ever-increasing number of discussions about pending credits over at SETI for proof of that... All you're doing is pacifying one group of complainers by shifting their complaint to another group.

Deadlines need to be reasonable, not excessively short or excessively long. Then it is up to us to keep up with the bouncing ball and set our resource allocations properly.

I respect that you're a tester, but you're not going to sway me on this.

BTW, yes, I knew I should've left this subject alone. People are far too sensitive to the EDF boogie monster...

Winterknight
Winterknight
Joined: 4 Jun 05
Posts: 1220
Credit: 311979618
RAC: 679316

I agree that deadlines should

I agree that deadlines should be reasonable, I also think Seti's at the moment are excessive.

As far as App's are concerned, Einstein is a main stream project, therefore it has to be expected most volunteers do install, setup and forget. They do not want to be manually installing the latest test version with app_info files etc.
I also don't think the 'install and forget' user would want to be changing their resource share every time a project changes its application. In fact I know BOINC participants that haven't even looked at changing resource share, each project is still in default location and shares are 100:100:.....:100 and virtually never visit these boards.

The times I see from my unit partners would indicate that even early P4's will struggle to complete within the deadline if only used in office hours, or evenings and w/end, even if they are only attached to Einstein.

Any one with a P3 or lower, still crunching here probably runs extended hours and must be considered a serious cruncher.

Think my Seti2 to 3 times went from 8 to 20+ hrs, but as you say memory is a bit fuzzy.

Brian Silvers
Brian Silvers
Joined: 26 Aug 05
Posts: 772
Credit: 282700
RAC: 0

RE: I agree that deadlines

Message 75307 in response to message 75306

Quote:
I agree that deadlines should be reasonable, I also think Seti's at the moment are excessive.

Whew... SETI needs to go back and reduce their deadlines. As I mentioned over there, I think they're going to see some database pains in the next few weeks as the first real long multibeam deadlines begin expiring. For those not up on what's going on over at SETI, there are some deadlines out there now that are into February of 2008. I have 3 tasks that I downloaded a few hours ago that have deadlines of Jan 27th. With luck, the reissues for the results expiring in late November will be picked up by faster or "better" hosts, meaning the new hosts actually turn stuff in rather than pick stuff up and let it languish.

Quote:

As far as App's are concerned, Einstein is a main stream project, therefore it has to be expected most volunteers do install, setup and forget. They do not want to be manually installing the latest test version with app_info files etc.

Bernd has been pretty good with promoting the apps from beta to production as they are proven to be stable. Yeah, it's still somewhat a public beta and it still bothers me a bit that it's being done this way rather than how SETI Beta works, but the super-nasty bugs in the Windows app (Windows is the dominant platform) that existed with S5R2 and the validator problems that were also with S5R2 have been taken care of for some time now. The DLL-loading issue seems to be heading towards something that can't be taken care of by a science application unless a science application forces a reboot, meaning it is not an Einstein issue, but is a heap issue with the system and no BOINC project will work until a reboot is performed and the user is likely to note other problems with their system anyway.

Quote:

I also don't think the 'install and forget' user would want to be changing their resource share every time a project changes its application.

No, but just like the transition of SETI 2.x to 3.x and SETI-BOINC to SETI-BOINC-Enhanced, certain minimums will gradually need to be increased over time so that better science is performed. Astro's P60 can't do any seriously significant contribution at SETI even with the help of that special app that was created.

Quote:

The times I see from my unit partners would indicate that even early P4's will struggle to complete within the deadline if only used in office hours, or evenings and w/end, even if they are only attached to Einstein.

There was a P4 1.5GHz host as a wingman on one of my results. It took 140,099 seconds or about 38.92 hours. Since that time, they have posted a higher runtime of 165,025 seconds (45.84 hours). So, based on that, yes, they would have issues if only being run "office hours" at 100% resource share. That would be a good example of a tight deadline and perhaps a borderline short deadline. The host is turning stuff back in at about the same rate as the runtimes, so it is being used at or near 24/7.

Quote:

Any one with a P3 or lower, still crunching here probably runs extended hours and must be considered a serious cruncher.

Agreed, but P3/AthlonXP and higher will be greatly helped by the SSE optimization in the same area as the optimization that's in the MacOS X app, thus relieving some of the deadline pressure.

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1364
Credit: 3562358667
RAC: 1580

RE: I joined LHC during

Message 75308 in response to message 75297

Quote:
I joined LHC during the largest SETI outage in the project's history, so the 1% was designed to be a "if neither SETI nor Einstein has work, get some LHC work".

LHC really isn't the best project to use as a backup, it's WU generation is very eratic, and is often out of work entirely.

http://boincstats.com/stats/project_graph.php?pr=lhc

Brian Silvers
Brian Silvers
Joined: 26 Aug 05
Posts: 772
Credit: 282700
RAC: 0

RE: LHC really isn't the

Message 75309 in response to message 75308

Quote:

LHC really isn't the best project to use as a backup, it's WU generation is very eratic, and is often out of work entirely.

I know. I forgot to mention that at that time there was no other project that I wanted to contribute time to and that I was perfectly fine with going idle. That being said, the graph there at BOINCstats is a bit deceptive. When I joined in January, there was some work here and there. The XML export for the stats didn't happen until you see the spike in June. From what I remember, I got some work, enough to make #1 in the start day until they started issuing more over the past few months. It was also enough to tide me over for a little while, but I still ended up with downtime as Einstein was still down and either SETI went down or work generation was spotty, or I just got cranky with SETI and decided to go idle ;)

However, LHC has started to generate work on a somewhat consistent basis now. That's why you'll note I have non-zero RAC in that project...

gaz
gaz
Joined: 11 Oct 05
Posts: 650
Credit: 1902306
RAC: 0

running vista on dial up

running vista on dial up getting 14 days deadline and a run time of 25hrs per unit run 24/7 most of the time only problem i can see is the next ready to run as the same deadline not a good idea if i have to go to a lot shoter run time

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1364
Credit: 3562358667
RAC: 1580

RE: The XML export for the

Message 75311 in response to message 75309

Quote:
The XML export for the stats didn't happen until you see the spike in June. From what I remember, I got some work, enough to make #1 in the start day until they started issuing more over the past few months.

I know about the busted XML export, and was only looking at the last 60 days graph. LHC only looks to've been at at most a 50% duty cycle the last month, and completely down the month prior.

Brian Silvers
Brian Silvers
Joined: 26 Aug 05
Posts: 772
Credit: 282700
RAC: 0

RE: LHC only looks to've

Message 75312 in response to message 75311

Quote:
LHC only looks to've been at at most a 50% duty cycle the last month, and completely down the month prior.

Not exactly. I mean you're right about not being active in September, but they did some interesting limits on quotas in October. They made an annoucement in the UK about the project and so they built up a large batch of work, then restricted everyone to 2/core (4 core max), with the idea that they wanted new participants to have work so that they wouldn't be discouraged. That meant that no system out there would pull any more than 8 results per day. That went on for a week or so, then they doubled it (4/core, 4 core max). That went on for a while, then they bumped to the current level of 10/core with 4 core max.

See, over there there was whinging about "fair distribution" of work. The prior administrators had set quotas up at 500/core. People would complain about not getting any work while they saw systems with hundreds of results.

We volunteers are a complaining lot, aren't we?

Bob
Bob
Joined: 23 Nov 05
Posts: 6
Credit: 106294177
RAC: 406676

Since I started this current

Since I started this current ruckus, my name was taken in vain (just kidding), and the thread was still open in a browser window...

From Jim:

Quote:
I'm sorry, your logic on this totally escapes me. You set the resource share. Did you take into consideration the longer run times and shorter deadlines with respect to Einstein when you set it up? Would it have been better for Boinc to let the Einstein WU die by strictly following your specific resource share? Then you could have returned a dead WU and not received any credit for doing 61 hours of work.

From Brian:

Quote:
There is though a culture that exists, particularly with SETI, and perhaps specifically with David Anderson, that believes in "set and forget". That's what Bob seems to want to do, set the allocations and forget them. That's fine, so long as he also "forgets" to watch the day-to-day operation. It appears that he still wants to micro-manage.

I am, by any definition, a casual volunteer. I drop by message boards occasionally every 6-8 weeks on the average if I notice a problem (this thread has taken my time allocation for the next couple of years). The disparity I noticed is mentioned at the beginning of the thread.

After being a SETI casual volunteer for a long time, when I switched to BOINC I decided to give some of the spare cycles to a couple of other projects. I picked Einstein and Rosetta, and gave them each a 10% resource share. That’s all the thought that went into it. My assumption was that the BOINC scheduler would make do with the allocation, especially since my computers easily met the BOINC minimum requirements and the intent is to use spare cycles anyway. Also note that when you join a project, there’s nothing that says your computer needs to be this tall to ride.

Just because I’m a casual volunteer and this is not my hobby doesn’t mean I am irresponsible. I make sure that the workunits assigned to my computer are completed and returned. As a responsible casual volunteer, allowing a workunit to consume 61 hours of cycles and then fail screws both the project and the buddy system. So I raised a red flag, even though EDF handled things. If trying to be responsible is micro-managing, guilty. Software is built by people, and without feedback nothing gets fixed, especially on a project with this many computers involved.

Finally, for the record I'm actually quite please the BOINC developers had the foresight to include EDF in the scheduler. What I was trying to point out was that picking a deadline that causes EDF to be exercised can have unforseen consequences, much as running an engine at redline for extended periods. I realize I will never convince some people of this and won't try.

Brian Silvers
Brian Silvers
Joined: 26 Aug 05
Posts: 772
Credit: 282700
RAC: 0

RE: My assumption was that

Message 75314 in response to message 75313

Quote:
My assumption was that the BOINC scheduler would make do with the allocation, especially since my computers easily met the BOINC minimum requirements and the intent is to use spare cycles anyway. Also note that when you join a project, there’s nothing that says your computer needs to be this tall to ride.

It is making do with the allocation. It is just not 100% spot-on when viewed over a short timeframe.

Quote:

What I was trying to point out was that picking a deadline that causes EDF to be exercised can have unforseen consequences, much as running an engine at redline for extended periods. I realize I will never convince some people of this and won't try.

I'm trying to not be offensive, but I do understand EDF, quite well actually. What you're not looking at is that your resource allocation plays a role. The requirements of the projects are not always going to be static across multiple years. Generally speaking, a space/physics-based project is going to want to expand their science level as processing power grows. SETI has done this in the past, with SETI Classic (version 2.x to 3.x), then to SETI BOINC, then to SETI BOINC Enhanced, to now multibeam.

When you joined, results ran substantially quicker. They had a 2 week deadline which was doable at 10% for you. Life was good. With S5R2, that all changed. The project made a poor decision, IMO, of not increasing deadlines out further than they did. They still have them a bit too short, but relief should be on the way with the SSE optimization. That's going to help anyone with a P3 or Athlon XP and newer. This will take care of the bulk of the user base.

Finally, I think the fact that SETI's deadlines are now so excessively long is distorting your take on this. If you look at what I'm saying, you'll see I do indeed understand you, I just don't fully agree that Einstein is "behaving badly".

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.