Mac 10.6 CUDA app "Ready to Start" not starting

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0
Topic 194777

So I've got the BOINC manager prefs set to "use GPU always" and "run always" and my prefs at einstein@home are set to run GPU whenever even if there is activity but both of my systems which have run GPU units in the past are just running CPU jobs only. Einstein@home is my only project. I have a boatload of CUDA APB2cuda23 jobs sitting in my queue "ready to start". I'm using BOINC 6.10.33 on OS X 10.6.2.

Any help would be greatly appreciated! Thanks!

-DougB
Office of CyberInfrastructure/National Science Foundation

Gundolf Jahn
Gundolf Jahn
Joined: 1 Mar 05
Posts: 1079
Credit: 341280
RAC: 0

Mac 10.6 CUDA app "Ready to Start" not starting

How many GPU memory is available? The newer BOINC clients don't start GPU tasks if there's not enough free. During download the total memory is checked.

Gruß,
Gundolf

Computer sind nicht alles im Leben. (Kleiner Scherz)

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0

RE: How many GPU memory is

Message 97085 in response to message 97084

Quote:

How many GPU memory is available? The newer BOINC clients don't start GPU tasks if there's not enough free. During download the total memory is checked.

Gruß,
Gundolf

Interesting. I'll check that out. When you say "newer" which versions do you mean?

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5872
Credit: 117749565467
RAC: 34896836

RE: RE: How many GPU

Message 97086 in response to message 97085

Quote:
Quote:

How many GPU memory is available? The newer BOINC clients don't start GPU tasks if there's not enough free. During download the total memory is checked.

Gruß,
Gundolf

Interesting. I'll check that out. When you say "newer" which versions do you mean?


I believe this feature was first added in 6.10.25 - so quite recent.

Check out the CUDA app sticky thread. I think you'll find other reports of Mac users being caught by this - enough total memory to download but not enough available to start crunching because of what the system is using. Maybe you can somehow shut down the GUI when you finish normal use and free up enough graphics memory to allow the memory requirement to be met. Before 6.10.25 I believe the tasks would immediately error out rather than simply not ever start.

EDIT: Here is a message in the sticky thread which comes from a user with a Macpro who had the same problem. If you read on from that point you will get further thoughts about the problem, including a post from Oliver, one of the Devs.

Cheers,
Gary.

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2143
Credit: 2960399375
RAC: 709462

Oliver's post in the ABP1

Oliver's post in the ABP1 thread (now unsticky) was even more explicit.

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 984
Credit: 25171438
RAC: 23

Hi Doug, There are two

Hi Doug,

There are two things you want to check to find out whether the amount of available GPU memory might be the problem as suggested by Gundolf:

1) Does the BOINC client log contain the following message?

Can't get available GPU RAM:

If yes, the client can't determine the available memory in which case it won't use the GPU.

2) Please enable [pre][/pre] in the BOINC client configuration, restart the client and check whether the log contains the following message:

[cpu_sched_debug] : insufficient GPU RAM (MB MB)

If this is the case please report back the x-value.

Cheers,
Oliver

Einstein@Home Project

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0

RE: Hi Doug, There are two

Message 97089 in response to message 97088

Quote:

Hi Doug,

There are two things you want to check to find out whether the amount of available GPU memory might be the problem as suggested by Gundolf:

1) Does the BOINC client log contain the following message?

Can't get available GPU RAM:

If yes, the client can't determine the available memory in which case it won't use the GPU.

2) Please enable [pre][/pre] in the BOINC client configuration, restart the client and check whether the log contains the following message:

[cpu_sched_debug] : insufficient GPU RAM (MB MB)

If this is the case please report back the x-value.

Cheers,
Oliver

Yes, this looks like to be the issue, on the Imac I'm using the following comes back.

Fri Feb 19 08:36:08 2010 Einstein@Home [cpu_sched_debug] p2030_54024_84455_0105_G69.60-01.71.C_1.dm_72_1: insufficient GPU RAM (387MB < 400MB)

The card is an 8800GS with 512Mb of video ram (reported in profiler)

on the 8 core mac pro we have with an 8800GT and 512Mb of GPU memory the error is similar:

Fri Feb 19 08:44:57 2010 Einstein@Home [cpu_sched_debug] p2030_54024_82624_0090_G66.00-01.98.C_2.dm_164_0: insufficient GPU RAM (365MB < 400MB)

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0

RE: RE: Hi Doug, There

Message 97090 in response to message 97089

Quote:
Quote:

Hi Doug,

There are two things you want to check to find out whether the amount of available GPU memory might be the problem as suggested by Gundolf:

1) Does the BOINC client log contain the following message?

Can't get available GPU RAM:

If yes, the client can't determine the available memory in which case it won't use the GPU.

2) Please enable [pre][/pre] in the BOINC client configuration, restart the client and check whether the log contains the following message:

[cpu_sched_debug] : insufficient GPU RAM (MB MB)

If this is the case please report back the x-value.

Cheers,
Oliver

Yes, this looks like to be the issue, on the Imac I'm using the following comes back.

Fri Feb 19 08:36:08 2010 Einstein@Home [cpu_sched_debug] p2030_54024_84455_0105_G69.60-01.71.C_1.dm_72_1: insufficient GPU RAM (387MB ". Just the second one, but I'll try to grep for the error to make sure I did not miss it.

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 984
Credit: 25171438
RAC: 23

Hi Doug, RE: Oh and

Message 97091 in response to message 97090

Hi Doug,

Quote:
Oh and I did not see the first error message "Can't get available GPU RAM: ". Just the second one, but I'll try to grep for the error to make sure I did not miss it.

You don't have to as both cases are (well, should be) mutually exclusive.

Anyway, unfortunately your standard desktop setup consumes too much memory such that there's not enough free memory available for our CUDA app which currently requires the 400 MB stated in the error message. Are the GPU cards concerned used as (desktop) display devices? If so, I'd recommend to use only dedicated cards for crunching purposes - using display cards will also affect their performance, leading to a sort of sluggish graphics response.

Out of curiosity: if you use them as display devices, are they used in a dual-head setup and which resolutions do you run? Does the problem persist after a fresh reboot?

Thanks,
Oliver

Einstein@Home Project

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0

RE: Hi Doug, RE: Oh and

Message 97092 in response to message 97091

Quote:

Hi Doug,

Quote:
Oh and I did not see the first error message "Can't get available GPU RAM: ". Just the second one, but I'll try to grep for the error to make sure I did not miss it.

You don't have to as both cases are (well, should be) mutually exclusive.

Anyway, unfortunately your standard desktop setup consumes too much memory such that there's not enough free memory available for our CUDA app which currently requires the 400 MB stated in the error message. Are the GPU cards concerned used as (desktop) display devices? If so, I'd recommend to use only dedicated cards for crunching purposes - using display cards will also affect their performance, leading to a sort of sluggish graphics response.

Out of curiosity: if you use them as display devices, are they used in a dual-head setup and which resolutions do you run? Does the problem persist after a fresh reboot?

Thanks,
Oliver

Well, it's a fairly vanilla Imac. The only thing the card is doing is providing display on the primary Imac monitor. I'm not using any GL or GPU applications other than what MacOS uses for display acceleration (not even a 3d screensaver).

The current memory requirements are going to exclude many people who only have one GPU and are looking to number crunch opportunistically with it while they are not using it for anything other than typical 2d display stuff.

I'll have to do a reboot to see if the problem persists and how long it persists.

dbaggett
dbaggett
Joined: 16 Aug 06
Posts: 7
Credit: 13784313
RAC: 0

RE: RE: Hi

Message 97093 in response to message 97092

Quote:
Quote:

Hi Doug,

Quote:
Oh and I did not see the first error message "Can't get available GPU RAM: ". Just the second one, but I'll try to grep for the error to make sure I did not miss it.

You don't have to as both cases are (well, should be) mutually exclusive.

Anyway, unfortunately your standard desktop setup consumes too much memory such that there's not enough free memory available for our CUDA app which currently requires the 400 MB stated in the error message. Are the GPU cards concerned used as (desktop) display devices? If so, I'd recommend to use only dedicated cards for crunching purposes - using display cards will also affect their performance, leading to a sort of sluggish graphics response.

Out of curiosity: if you use them as display devices, are they used in a dual-head setup and which resolutions do you run? Does the problem persist after a fresh reboot?

Thanks,
Oliver

Well, it's a fairly vanilla Imac. The only thing the card is doing is providing display on the primary Imac monitor. I'm not using any GL or GPU applications other than what MacOS uses for display acceleration (not even a 3d screensaver).

The current memory requirements are going to exclude many people who only have one GPU and are looking to number crunch opportunistically with it while they are not using it for anything other than typical 2d display stuff.

I'll have to do a reboot to see if the problem persists and how long it persists.

The Cuda app started after a reboot. I have not a clue what could be taking up GPU memory inbetween the time a cuda app ends and a new one begins. Should it release all the memory it was using before proceeding onto the next app?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.