Unsent?
4 Sep 2017 11:58:32 UTC
Topic 209547
(moderation:
Why do I have so many pending tasks? When I checked the tasks the co-pilot job is usually "unsent" so I am sitting pretty at just my tasks with no wing man -Great. What a waste of computing time.
For some types of work,
)
For some types of work, Einstein is able to make many work units from just one set of (large) file downloads. So it makes sense to send that type of work to systems which already have downloaded the specific large files required.
That is one reason for the "unsent" phenomenon you mention. The wait is not indefinite, and in no sense is your computing time wasted.
Patience is all that is required of you.
Hi Jacqui, Welcome to the
)
Hi Jacqui,
Welcome to the forums.
Archae86 has already given you the most likely reason for seeing tasks in your list that are 'unsent' to a quorum partner machine. This is always seen to some extent near the start of a new search but I was a bit intrigued as to why you might be seeing it now when the current search is well established.
So I looked at your most prolific machine (highest RAC) which is doing GW tasks (which is the search that can show the problem) and saw that you currently have 109 tasks total of which 79 are validated and just 16 are pending. That seemed like a reasonably normal ratio so I decided to look at the 16 to see how many had quorum partners. I gave up after about 12 because all the ones I looked at did have partners and both tasks in each quorum were issued at about the same time. So, for that machine anyway, I saw no evidence of "co-pilot job is usually unsent".
My main reason for posting a reply is to give you a strategy for minimising "pendings", whatever the cause.
Pendings are a normal part of life. The normal causes are to do with how quickly your quorum partner returns their copy. They normally have two weeks before a task will "expire". If a person is careless enough to stop crunching (permanently or even temporarily - eg. away for a couple of weeks) without aborting and returning any work on hand, there is nothing the project can do to avoid the two week delay. There is a significant number of such tasks that are never returned. So, there will always be some multi-week pendings.
Some people like to keep plenty of work on hand. A couple of days worth is reasonable because there is always the potential for a project outage. Outages of more than a day or two are very rare here so it doesn't make much sense to set the preferences for anything more than say 3 days to protect against an outage late on Friday that can't be corrected until the following Monday. But there are people who set 10+ day caches. If such a person is your quorum partner, because of the effects of locality scheduling, you both may share lots of tasks based on the same set of large data files. You may be stuck with a situation where you get lots of 10+ day pendings if you have a very small work cache yourself and complete your work promptly.
You should realise that none of your long term pendings are wasted. You can shorten the wait for validation by not having a miniscule work cache setting yourself. You are not going to be able to change the anti-social tendency of some folk who set 10+ day caches (and you shouldn't do this yourself for lots of good reasons, not just the anti-social nature of it) but you can shorten the wait a little if you take the quite reasonable action of setting a 3 day cache. That way, when you get to crunch a task it will already be 3 days 'matured' :-) and you will effectively shorten the average wait time for a pending to be validated by about 3 days. Of course, there will then be others who complete their work quickly who can then complain about you "wasting their work" :-).
The setting of appropriate work cache size parameters is very important and there are lots of different factors to be considered. The average length of a pending task really ISN'T one of importance. There can be no single recommendation of a 'best' setting. It depends very much on the mix of work your machine handles. In general terms, relatively short is good, particularly if you contribute to multiple projects.
Cheers,
Gary.
Jacqui wrote:Why do I have so
)
At another project I have the same problem, but after 3 or 4 days of me crunching their workunits the 'pending' versus the 'unsent' ones have stabilized and they are holding where they are. The same thing should happen here, which is where a 3 day work cache may help both of us, mine right now is 1.5 days and I am doing workunits over there in about 70 seconds each. So I have LOTS of workunits waiting for my 'copilot', or 'wingman', to either get their workunit or finish them when they do.
I too, have a problem with a
)
I too, have a problem with a significant number of Unsent wingman jobs. I understand the logic of Archae86's answer but feel that that this is an insufficient answer. Exactly how big are these files? Why is it ok for me to send the server 50-100 Mb with no problem but not for the server to send 'big' files twice. ( I can only connect the PC in question every couple of days).
The reason I am concerned is that in the last 15days or so my machine (Peter-PC) has gone from having most of it's results quickly verified to having almost none verified. I have at this time 3 verified and 46 pending.Of the 3 that are verified 2 were because someone else had had to wait 8 days before I got sent them.
I have currently got 17 jobs waiting 10 days for a partner and still unsent and 20 waiting 7 days and still unsent.
I feel sure that eventually these jobs will get partners but in the interest of keeping the punters happy then I feel that the project administrators might consider resending these jobs after a shorter time and hang the expense of sending the big files again.Rather then waiting an indeterminate time for someone with the right dataset to come back.
Regards Peter
The datafiles for each task
)
The datafiles for each task are about 40-50MB which would need to be downloaded everytime a new task is assigned tou your host. Even if a similar task was just finished. In contrast the result files per task are usually around <6MB. Imagine that every host wants to download the same file over and over again and we have several 10.000 hosts what that would mean for our bandwidth.
The current rise of pending tasks is because the institutional computing grids that usually clear up the pending queue are busy doing other work. The work is still flowing and your computers are still busy searching for gravitational waves. A rising number of pending tasks is no sign of less work done it just means that the metric of contribution in terms of Credit will be a little bit delayed but that affects everyone.
Thank you for a more detailed
)
Thank you for a more detailed explanation Christian. Unfortunately that explanation has not satisfied my computer which has gone on strike. After brief power cut this afternoon the ssd in the computer has decided that it is broken; permanently I think, which is somewhat annoying. There may be a long delay now before I manage to resurrect it. Sorry to all the potential wingmen when those jobs get timed out.
Pete