Can I multi-thread the CPU tasks?

mikey
mikey
Joined: 22 Jan 05
Posts: 6,237
Credit: 550,290,859
RAC: 169,656
Topic 223173

I'm keeping the electic bill down this Summer but not running the gpu's but need to keep my RAC up to post, my question is can I multi-thread the cpu workunits to get thru them quicker? I have a basic app_config.xml file, from YoYo, so the name is wrong BUT if I fix the name will it work is my question?

<app_config>
<app_version>
<app_name>ecmp2</app_name>
<cmdline>-t 4</cmdline>
<avg_ncpus>4</avg_ncpus>
</app_version>
</app_config>

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 832
Credit: 751,780,283
RAC: 1,348,947

Einstein doesn't have any

Einstein doesn't have any multi-thread apps.  Just single thread.  The only project I am familiar with with MT apps is Milkyway N-body application.

You could try and run doubles at least on each gpu since all we have to crunch now is Gamma-Ray gpu tasks.  They are easy on the memory needs. Not like the heavy memory usage of the Gravity Wave app.

But since you've said you want to exclude gpu use, all you can do is put more cores to work for you on more tasks.

 

mikey
mikey
Joined: 22 Jan 05
Posts: 6,237
Credit: 550,290,859
RAC: 169,656

Keith Myers wrote: Einstein

Keith Myers wrote:

Einstein doesn't have any multi-thread apps.  Just single thread.  The only project I am familiar with with MT apps is Milkyway N-body application.

You could try and run doubles at least on each gpu since all we have to crunch now is Gamma-Ray gpu tasks.  They are easy on the memory needs. Not like the heavy memory usage of the Gravity Wave app.

But since you've said you want to exclude gpu use, all you can do is put more cores to work for you on more tasks.

That's not good, I may have to fire up a gpu for a day or so to bring the RAC back up then, I'm getting 693 credits for each workunit and they are taking around 12k seconds to run one...hmmm. I'm running the "Gamma-ray pulsar search #5" tasks.

adrianxw
adrianxw
Joined: 21 Feb 05
Posts: 211
Credit: 214,884,380
RAC: 161,737

Multithreading is an

Multithreading is an attribute that is built into the program and is suitable for certain types of work only, (applications where the process can run in parallel). You cannot multithread a program that was not written to be multithreaded.

I used to have a tutorial on how to write multithreaded applications, but it is not online anymore. It discussed the hows and whys. I used to have a fixed internet connection so could host the site on my own server, but where I live now, the connection is shared between the houses on the estate and cannot support server hosting.

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.

adrianxw
adrianxw
Joined: 21 Feb 05
Posts: 211
Credit: 214,884,380
RAC: 161,737

The way back machine has the

The way back machine has the simple tutorial here:

https://web.archive.org/web/20100215133324/http://www.adrianxw.dk/SoftwareSite/Threads/Threads1.html

It was a long time ago, but I doubt things have changed that much.

Another point to bear in mind is multithreaded tasks use more than one CPU, if you run a second workunit in the other CPU, you will get as much work done as a multithreaded single work unit.

Wave upon wave of demented avengers march cheerfully out of obscurity into the dream.

Eugene Stemple
Eugene Stemple
Joined: 9 Feb 11
Posts: 15
Credit: 46,901,204
RAC: 96,904

@Mikey I think what you're

@Mikey

I think what you're looking for is the <project_max_concurrent> tag in the app_config.xml file, and this goes at the top level - right after the opening <app_config> tag.  I set that to one less than the number of cores although one can set equal to number of cores if one can tolerate the consequent response lag.  There is also a tag for <max_concurrent> for each different application section.  I don't use the <avg_ncpus> tag so I think you can safely delete that line.  Yes, I'm seeing 3 to 4 hours per CPU task; I give 6 cores to Einstein, where one is usually supporting a GPU task and the remaining 5 run CPU tasks.  Sure, the GPU is ~10 times more productive, but 5 otherwise idle CPUs give half-a-GPU production and that's worthwhile in my mind.

 

mikey
mikey
Joined: 22 Jan 05
Posts: 6,237
Credit: 550,290,859
RAC: 169,656

Eugene Stemple

Eugene Stemple wrote:

@Mikey

I think what you're looking for is the <project_max_concurrent> tag in the app_config.xml file, and this goes at the top level - right after the opening <app_config> tag.  I set that to one less than the number of cores although one can set equal to number of cores if one can tolerate the consequent response lag.  There is also a tag for <max_concurrent> for each different application section.  I don't use the <avg_ncpus> tag so I think you can safely delete that line.  Yes, I'm seeing 3 to 4 hours per CPU task; I give 6 cores to Einstein, where one is usually supporting a GPU task and the remaining 5 run CPU tasks.  Sure, the GPU is ~10 times more productive, but 5 otherwise idle CPUs give half-a-GPU production and that's worthwhile in my mind.

Are you running 5 cpu tasks at one time or one cpu task using 5 cpu cores at one time? I want to do the latter as I can do the former right now they just take too long, compared to the gpu tasks, and I have the cpu cores available.

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 832
Credit: 751,780,283
RAC: 1,348,947

Quote:one cpu task using 5

Quote:
one cpu task using 5 cpu cores at one time? I want to do the latter

As mentioned earlier, you can't do that here at Einstein. There are no applications capable of that method.

 

mikey
mikey
Joined: 22 Jan 05
Posts: 6,237
Credit: 550,290,859
RAC: 169,656

Keith Myers wrote: Quote:one

Keith Myers wrote:

Quote:
one cpu task using 5 cpu cores at one time? I want to do the latter

As mentioned earlier, you can't do that here at Einstein. There are no applications capable of that method.

I was hopeful someone had found a way, but I knew that.

Eugene Stemple
Eugene Stemple
Joined: 9 Feb 11
Posts: 15
Credit: 46,901,204
RAC: 96,904

@Mikey wrote: Are you

@Mikey wrote:

Are you running 5 cpu tasks at one time or one cpu task using 5 cpu cores at one time? I want to do the latter as I can do the former right now they just take too long, compared to the gpu tasks, and I have the cpu cores available.

It is, as the expression goes, "six of one and half a dozen of the other."  I'm running 5 cpu tasks at one time (in 5 cores), so completing 5 tasks in 3 hours, at an average of 36 minutes each.  *IF* it were possible to run 1 task in 5 cores then presumably it would complete in 1/5 the time, i.e. 36 minutes, and doing that on a sequence of tasks I would complete 5 of them in 3 hours.  There is no free lunch.

mikey
mikey
Joined: 22 Jan 05
Posts: 6,237
Credit: 550,290,859
RAC: 169,656

Eugene Stemple wrote: @Mikey

Eugene Stemple wrote:

@Mikey wrote:

Are you running 5 cpu tasks at one time or one cpu task using 5 cpu cores at one time? I want to do the latter as I can do the former right now they just take too long, compared to the gpu tasks, and I have the cpu cores available.

It is, as the expression goes, "six of one and half a dozen of the other."  I'm running 5 cpu tasks at one time (in 5 cores), so completing 5 tasks in 3 hours, at an average of 36 minutes each.  *IF* it were possible to run 1 task in 5 cores then presumably it would complete in 1/5 the time, i.e. 36 minutes, and doing that on a sequence of tasks I would complete 5 of them in 3 hours.  There is no free lunch.

There is at some Projects but the programmers didn't make the workunits so they can be done that way here. Prime Grid workunits, the LLR ones, are also written so they can be multi-threaded and are significantly faster when you throw more cpu cores as a workunit. MIlkyWay wrote their current units from the beginning to allow multi-threading and they are much faster the more cpu cores you throw at them. And yes you can get a "free lunch" as you call it if the units fit into memory and the units are designed that way.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.