Many Errors

Bob Bainbridge
Bob Bainbridge
Joined: 3 Nov 10
Posts: 2
Credit: 140489560
RAC: 0
Topic 221793

I crunched Seti WU's for 19 years and did some Einstein 10 years ago. Now with Seti grinding to a halt I'm trying Einstein again. I'm getting a high error count on at least 3 computers with NVIDIA 570, 670 and 770 GPU's. At first glance I see EXIT STATUS 65 (0x00000041) unknown error code on Gamma-Ray Pulsar WU's and EXIT STATUS 114 (0x0000007w2) unknown error code on Gravitational Wave Search WU's. Any idea what my problem is? Thanx,

Bob B.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109382442826
RAC: 35971435

Bob Bainbridge wrote:I

Bob Bainbridge wrote:

I crunched Seti WU's for 19 years and did some Einstein 10 years ago. Now with Seti grinding to a halt I'm trying Einstein again. I'm getting a high error count on at least 3 computers with NVIDIA 570, 670 and 770 GPU's. At first glance I see EXIT STATUS 65 (0x00000041) unknown error code on Gamma-Ray Pulsar WU's and EXIT STATUS 114 (0x0000007w2) unknown error code on Gravitational Wave Search WU's. Any idea what my problem is? Thanx,

Bob B.

Hi Bob, Welcome back to Einstein!

The easiest way to get a better idea of the cause of a compute error is to pick a failed task in the tasks list for the host in question on the website.  If  you click the Task ID link for the failed task, it will take you to the full stderr.txt output that was sent back to the project for that task.

I've looked at some example errors from all 3 hosts and something like the following is pretty common for the error condition.  This one is from the 570 but I saw similar memory allocation failures on the other two as well:-

% Starting semicoherent search over f0 and f1.
% nf1dots: 41 df1dot: 2.512676418e-015 f1dot_start: -1e-013 f1dot_band: 1e-013
% Filling array of photon pairs
Error in OpenCL context: CL_MEM_OBJECT_ALLOCATION_FAILURE error executing CL_COMMAND_NDRANGE_KERNEL on GeForce GTX 570 (Device 0).

The 570 may be too old and have not enough available memory, 'available' being the key word.  If you happen to be tying up memory elsewhere or attempting to run multiple concurrent GPU tasks, you should adjust things to see if more available memory will solve the problem, particularly for the two more recent GPUs which do each have 2GB in total and really that should be quite sufficient to at least run tasks singly.

I use Linux and only AMD GPUs so don't have relevant experience with either Windows or your type of GPU.  It seems pretty clear though that you somehow need to make more memory available to the app in order to allow all tasks to complete.  As well as the huge number of failures, you do have validated tasks for the gamma-ray pulsar app so stable operation should be possible.  To test that, you should change prefs to allow that app only with single tasks until you get something that works reliably.  That app will probably have lower memory requirements than the GW app.

Hope this helps!

Cheers,
Gary.

WB8ILI
WB8ILI
Joined: 20 Feb 05
Posts: 45
Credit: 923985877
RAC: 232456

Bob - It could be a GPU

Bob -

It could be a GPU memory issue. You have your computers "hidden" so it is hard to make any comment on experience I have had.

You give numbers but are these GT or GTX?

 

Bob Bainbridge
Bob Bainbridge
Joined: 3 Nov 10
Posts: 2
Credit: 140489560
RAC: 0

Thanks for the reply. I'll

Thanks for the reply. I'll take a look at my settings. I also started running World Community Grid but that wasn't using any GPU's so I started Einstein.

Bob B.

Richie
Richie
Joined: 7 Mar 14
Posts: 656
Credit: 1702989778
RAC: 0

Your computers are not

Your computers are not hidden, settings are fine in that regards.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109382442826
RAC: 35971435

WB8ILI wrote:... You have

WB8ILI wrote:
... You have your computers "hidden" ...

Did you actually check that??  I saw all the standard details of his hosts before composing my reply.  I checked again just now in case something had changed.  Nope.  His computers are perfectly visible, including the details you asked about.

Cheers,
Gary.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.