Errors with GravWave Search - GPU

halfempty
halfempty
Joined: 3 Apr 20
Posts: 14
Credit: 37595576
RAC: 0

Keith Myers wrote:You need to

Keith Myers wrote:

You need to write an app_config file for Einstein excluding the 3GB card from the project.  The instructions can be followed at the reference document page.

https://boinc.berkeley.edu/wiki/Client_configuration#Application_configuration

Glad to see a familiar face here. Found the <exclude_gpu> option in the cc_config section on that page, and may have even gotten it right. From the event log:

4/16/2020 12:03:27 AM | Einstein@Home | Config: excluded GPU.  Type: all.  App: all.  Device: 1

Now I'll have to wait and see, I still have about 10 hours left in the penalty box.

Betreger
Betreger
Joined: 25 Feb 05
Posts: 992
Credit: 1593185673
RAC: 777658

An odd fact is that one of

An odd fact is that one of myGTX10603GB has not gotten 1 of these poison pills since the 4th whereas the other has 47 since this latest run has started, Both are running 1 at a time.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5872
Credit: 117741925515
RAC: 34864523

Betreger wrote:An odd fact is

Betreger wrote:
An odd fact is ....

If you look at your tasks list (either in BOINC Manager or on the website) you'll find that the 'good' machine doesn't have any VelaJr1 tasks yet.  It's still processing resends for the previous pulsar G34731.  So, nothing odd there.

Don't worry, your luck will probably change soon enough :-).

Let us know if/when that happens.  Further confirmation that nvidia 3GB cards can't handle these tasks might help to get something done about it.

Cheers,
Gary.

Zalster
Zalster
Joined: 26 Nov 13
Posts: 3117
Credit: 4050672230
RAC: 0

So I moved a pair of 2080

So I moved a pair of 2080 Supers over to test these Gravity Waves. I've run 235 so far and no errors on any of them.  Running single task per card. Average time 7.4 minutes. All have been these VelaJr tasks. Unfortunately only 8 have validated as all the rest have yet to be sent to anyone. My wingman for the first 8 gave up on them and moved his systems to Gamma Rays. He is running a Nvidia 1660 card(6 GB of memory)

Alexander Favorsky
Alexander Favorsky
Joined: 18 Jun 16
Posts: 36
Credit: 176635468
RAC: 76550

I had that problem with

I had that problem with VelaJr1 tasks on my 2GB card too but now I'm doing them fine. The frequency is the same for errored out tasks and successfully finished ones. Doing one at a time.

Zalster
Zalster
Joined: 26 Nov 13
Posts: 3117
Credit: 4050672230
RAC: 0

Alexander Favorsky wrote:I

Alexander Favorsky wrote:
I had that problem with VelaJr1 tasks on my 2GB card too but now I'm doing them fine. The frequency is the same for errored out tasks and successfully finished ones. Doing one at a time.

 

Looks like 1 of yours just errored out. I think it's like Garry said. Certain Frequency seem to cause the errors versus others.  

https://einsteinathome.org/workunit/450508339

Richie
Richie
Joined: 7 Mar 14
Posts: 656
Credit: 1702989778
RAC: 0

I read they tuned the

I read they tuned the scheduler to require enough VRAM on the GPU for GW tasks to fit in properly. My host with 2GB GTX 960 however still got the same VelaJr1 type tasks that error out with CL_MEM_OBJECT_ALLOCATION_FAILURE like they did yesterday.

Alexander Favorsky
Alexander Favorsky
Joined: 18 Jun 16
Posts: 36
Credit: 176635468
RAC: 76550

Zalster wrote:Looks like 1 of

Zalster wrote:
Looks like 1 of yours just errored out.

Not 1. I got 74 errors in total, 11 of those are download errors and others are computation errors.

https://einsteinathome.org/host/12298595/tasks/6/0

halfempty
halfempty
Joined: 3 Apr 20
Posts: 14
Credit: 37595576
RAC: 0

Just a quick update to let

Just a quick update to let everyone know that after I excluded the 1060 3GB from Einstein there have been no more errors. Every VelaJr1 task on the 1060 3GB failed, every one on the 1050Ti 4GB or the 1660 6GB succeeded.

 

Zalster
Zalster
Joined: 26 Nov 13
Posts: 3117
Credit: 4050672230
RAC: 0

That supports my thought that

That supports my thought that a Nvidia GPU requires a minimum of 4 GB (27% usable OpenCl) for VelaJr GW work units.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.