Waiting for Memory - Practical use of Profiling

TiddlesHeisenberg
TiddlesHeisenberg
Joined: 10 Mar 15
Posts: 1
Credit: 7145186
RAC: 16076
Topic 208440

This "Wish" for the Wish-list isn't for my benefit but for the Einstein@Home project's productivity.

I realise a previous poster "Robert" 2 Mar 2016 21:57:26 UTC had a topic:

Avoid clogging BOINC by suspending instead of "Waiting for memory"

This was about a 4-core/8-thread cpu trying to run 8 tasks each asking for 400MB & the topic was met with some confusion.

My topic is similar:  if you try to run several tasks simultaneously, each of which needs a large amount of RAM, you can find that your computer simply doesn't have that much RAM available & tasks wait for memory.

My suggestion for Einstein admin is that, since the setup process for Einstein@Home involves profiling the capabilities of each new computer, the distribution of tasks to that computer could make good use of the profile.

In the 4-core/8-thread Robert example 8 tasks of 400MB would expect 3.2GB of RAM which an older laptop might not have available.  In my case some current Einstein tasks want ~1.24GB of RAM - the equivalent of 3 of the "large" 400MB Robert tasks.  I normally allocate 2 of my 4 cores for Einstein & my computer will happily run two 1.24GB tasks.  I did one day allocate all 4 cores to Einstein & only 3 Einstein tasks run using 3.76GB RAM while a 4th task is left with the status "Waiting for Memory".  So, like Robert, we can have cores free but doing nothing while a task waits for RAM that will never be available.

From Einstein's point of view then a better distribution of tasks to my computer would be to allow for the fact that my computer profile suggests I wouldn't be able to run 4x 1.24GB RAM tasks & to add to my schedule some tasks requiring cpu-time but not much RAM - that would also have solved Robert's problem.

So the Wish (suggestion) is for smarter scheduling using the computer profiles that Einstein already has.

I realise that is easy to say but tricky to do:  it suggests having a smarter scheduler running on the client computer in order to spot the need for a different kind of task (a "Waiting for memory" message that doesn't clear within "a reasonable time"...), suspend the waiting job & order up something more appropriate from the task list at Einstein HQ to avoid "wasting" all those cycles on the empty cpu core.

My solution to this was not to buy more RAM (which would be good) but, originally entirely unintentionally, usually to run a different @Home project on the other 2 cores that is cpu intensive but not a big RAM user.

Christian Beer
Christian Beer
Moderator
Joined: 9 Feb 05
Posts: 595
Credit: 59073284
RAC: 74785

This is a wish that is on the

This is a wish that is on the wishlist of every project. But there is little that projects can do about that since the decision what is actually run is made on the client side. The logic that determines what task is run and what project is asked for work and the information that is supplied during such a work fetch needs to be adjusted to allow a more complex logic. Taking into account that you can have multiple projects on one PC. This is a huge task where the impact it has on general throughput is unknown.

Since BOINC became a community project, development in the Client is done by volunteers and nobody so far showed interest in working on this part of the Client.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.