New Einstein@Home Radio Pulsar Search and NVIDIA GPU Code

Einstein@Home is beginning a new round of searching for radio pulsars in short-orbital-period binary systems.

This is accompanied by the release of a new application (called BRP3). The new application is particularly efficient on NVIDIA Graphics Processor Cards (up to a factor of 20 faster than the CPU-only application). In addition, when running on an NVIDIA GPU card, this new application makes very little use of the CPU (typically around 20% CPU use when the GPU is devoted to Einstein@Home).

The NVIDIA GPU application is initially available for Windows and Linux only. We hope to have a Macintosh version available soon. Due to limitations in the NVIDIA drivers, the Linux version still makes heavy use of the CPU. This will be fixed in Spring 2011, when a new version of the NVIDIA Driver is released. Many thanks to NVIDIA technical support for their assistance!

Because we have exhausted the backlog of data from Arecibo Observatory, this new application is being shipped with data from the Parkes Multibeam Pulsar Survey (from the Parkes Radio Telescope in Australia). In the next weeks we expect to also start using this new application on fresh Arecibo data taken with the latest 'Mock Spectrometer' back-end.

Questions, problems or bug reports related to this new application and search should be reported in this news item thread as a 'Comment'.

Bruce Allen
Director, Einstein@Home

Comments

Stephen W
Stephen W
Joined: 31 Oct 09
Posts: 3
Credit: 36529173
RAC: 0

New Einstein@Home Radio Pulsar Search and NVIDIA GPU Code

I'm running an i7 with two nvidia 470 gtx video card and Linux which seem to be getting computational errors on the new process. They seem to run ok till the end and then error out.

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: I'm running an i7 with

Quote:
I'm running an i7 with two nvidia 470 gtx video card and Linux which seem to be getting computational errors on the new process. They seem to run ok till the end and then error out.

Please provide some details so we can look into the problem. It'd be best to post links to the results/tasks that failed.

Thanks,
Oliver

 

Einstein@Home Project

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

I assume it's these

I assume it's these ones?

http://einsteinathome.org/host/3803628/tasks

Director, Einstein@Home

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: I assume it's these

Quote:

I assume it's these ones?

http://einsteinathome.org/host/3803628/tasks

Ok, looking at the error descriptions it seems that your BOINC client installation doesn't work properly, most likely to due to missing privileges (it has affected almost all E@H tasks on your host, it's not specific to the Radio Pulsar Search application). I recommend you reinstall the BOINC software, probably using the one that comes with your Linux distribution.

If that doesn't work please consider opening a separate thread in the Problems and Bug Reports board.

Hope this helps,

Oliver

 

Einstein@Home Project

Stephen W
Stephen W
Joined: 31 Oct 09
Posts: 3
Credit: 36529173
RAC: 0

Yes

Yes

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

Update: please check that

Update: please check that your host is able to run 32 bit applications. Right now it seems that it can only run 64 bit apps.

Oliver

 

Einstein@Home Project

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 556

RE: I'm running an i7 with

Quote:
I'm running an i7 with two nvidia 470 gtx video card and Linux which seem to be getting computational errors on the new process. They seem to run ok till the end and then error out.

It looks like the runtime on your tasks are showing as 0. You might want to check to see if there are any missing dependencies with these commands:

cd projects/einstein.phys.uwm.edu
ldd einsteinbinary_BRP3_1.06_i686-pc-linux-gnu__BRP3cuda32fullCPU

This should show all required libraries and if any are missing. Then also check to make sure IA32 support is enabled in kernel if running on a 64-bit system.

Stephen W
Stephen W
Joined: 31 Oct 09
Posts: 3
Credit: 36529173
RAC: 0

I tried reinstalling boinc

I tried reinstalling boinc and that did not work. It may have made things worse. So since the machine is only running boinc projects I reinstalled ubuntu and the nvidia drives then bionic. So far things are looking good.thanks for the help.

edjcox
edjcox
Joined: 10 Dec 05
Posts: 11
Credit: 141337850
RAC: 0

Can you provide a link to the

Can you provide a link to the new GPU code so I may run it?

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2139
Credit: 2752771467
RAC: 1446753

RE: Can you provide a link

Quote:
Can you provide a link to the new GPU code so I may run it?


Your Intel machine is running it already.

Ascholten
Ascholten
Joined: 20 Oct 10
Posts: 63
Credit: 2848108
RAC: 0

First I want to say thank

First I want to say thank you. Finally the GPU code is stable enough that I can run it on my machine without having to take one of the processors off crunching the CPU code to handle feeding GPU. Finally the code is stable enough that I dont have to cripple my vid card to have it not error out trying to run one of these.

I do have a question though. Currently on my GTX570 it is only running at about 35 percent GPU utilization. Are there any plans on improving the code to run at higher utilization rates hence more work for the time from the GPU's. Some projects, like milky way for example run in the upper 90's. It'd be nice to see einstein being able to use most of what the card can give us to crunch the numbers.

If there are tweaks or something else I can do that makes a significant difference in utilization (ie more than just a few percent) can someone please share them so I can try.

Thank you
Aaron

If god meant for us not to BOINC he'd have made our #$%^%^ shorter!!

Rechenkuenstler
Rechenkuenstler
Joined: 22 Aug 10
Posts: 138
Credit: 102567115
RAC: 0

What you can do is using a

What you can do is using a app_info.xml file run 3 GPU tasks in parallel. That should increase the GPU usage to ~90%

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: Currently on my GTX570

Quote:
Currently on my GTX570 it is only running at about 35 percent GPU utilization.

That might be influenced by how much your CPU cores are utilized and how much GPU memory is used by other things (e.g. your desktop). Our tests show that a single instance of the GPU app uses up to 73% when there are no other tasks running (remember: it still depends on approx. 5-30% of a CPU core, depending on your hardware) as the CPU still needs to feed the GPU.

Cheers,
Oliver

 

Einstein@Home Project

Rechenkuenstler
Rechenkuenstler
Joined: 22 Aug 10
Posts: 138
Credit: 102567115
RAC: 0

My experience shows, that the

My experience shows, that the GPU usage depends on the capacity of the vid card.

The more powerful the card, the less the usage of CPU capacity by one BRP task.

Example:

Quattro FX 770M has ~80% CPU usage with 210 minutes to finish
GT240 has ~75% with 145 minutes to finish
GTX240 has ~60% with 90 minutes to finish
GTX460 has ~50% with 55 minutes to finish

This results are more or less independend from the CPU usage, since BRP version 1.05

That means, that one BRP task can't create more GPU usage at the moment. So the alternative is to run more than one BRP tasks at the same times.

Ascholten
Ascholten
Joined: 20 Oct 10
Posts: 63
Credit: 2848108
RAC: 0

Thanks Rechen. Where is this

Thanks Rechen. Where is this file located at, the app into file? Granted it's early and I havent had my fill of coffee yet but I was not able to find it.

Thanks
Aaron

If god meant for us not to BOINC he'd have made our #$%^%^ shorter!!

tolafoph
tolafoph
Joined: 14 Sep 07
Posts: 122
Credit: 74659937
RAC: 0

Look here

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

RE: Due to limitations in

Quote:
Due to limitations in the NVIDIA drivers, the Linux version still makes heavy use of the CPU. This will be fixed in Spring 2011, when a new version of the NVIDIA Driver is released.

excuse me, what version are you talking about? 260.xxx and 270.xxx beta are available already.

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: excuse me, what

Quote:

excuse me, what version are you talking about? 260.xxx and 270.xxx beta are available already.

Well, the 260.xx (CUDA 3.2) drivers exhibit an issue that get's fixed in the next stable driver release (acknowledged/confirmed by NVIDIA). We don't expect our volunteers to run beta drivers (i.e. 270.xx).

Cheers,
Oliver

 

Einstein@Home Project

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

so, this high CPU usage under

so, this high CPU usage under CUDA crunching depends not only on drivers?

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: so, this high CPU usage

Quote:
so, this high CPU usage under CUDA crunching depends not only on drivers?

The root cause is the driver, but there's a workaround in our code that makes it possible to run the code, at all, without crashing. This mandatory workaround leads to the high CPU usage. As soon as NVIDIA released a fixed stable driver we'll remove this workaround, reducing the CPU usage to what's really required.

Keep in mind: this workaround (and driver issue) concerns Linux only!

Oliver

 

Einstein@Home Project

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

If ever will be possible to

If ever will be possible to crunch on ATI GPU? if believed to specs it has much more calculating power.

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: If ever will be

Quote:
If ever will be possible to crunch on ATI GPU? if believed to specs it has much more calculating power.

FYI...

 

Einstein@Home Project

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 556

RE: RE: so, this high CPU

Quote:
Quote:
so, this high CPU usage under CUDA crunching depends not only on drivers?

The root cause is the driver, but there's a workaround in our code that makes it possible to run the code, at all, without crashing. This mandatory workaround leads to the high CPU usage. As soon as NVIDIA released a fixed stable driver we'll remove this workaround, reducing the CPU usage to what's really required.

Keep in mind: this workaround (and driver issue) concerns Linux only!

Oliver

NVIDIA has a new driver on their site, version 260.19.36. Would this driver be the one that fixes the above issue or is there a later fix planned by NVIDIA? Thanks.

telegd
telegd
Joined: 17 Apr 07
Posts: 91
Credit: 10212522
RAC: 0

I notice that Ubuntu has the

I notice that Ubuntu has the 270-beta in one of the repos.

However, I think the E@H coders have to rewrite the Linux app for the new driver and they probably won't do that until the new driver is in final form and released officially.

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

and what's wrong with mac?

and what's wrong with mac? there is still no gpu app for it.

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686056996
RAC: 576866

I think it's wise to wait

I think it's wise to wait until the current cross-validation issues for the CUDA app are completely solved before introducing another platform.

As for the beta driver: Hmm... I'm not sure whether this beta already contains the fix for the driver problem. I'll check.

If so....hmm....maybe it would be a nice idea to release an app that could be triggered to disable the driver-bug workaround, e.g. by creating a special named file in the BOINC E@H project directory as was done for the GW app to enable some debugging features iirc. This way, those who are brave enough to use a beta driver could already beta-test the Linux app in the state it will be for the new official driver.

Oliver, what do you think?

CS
Heinz

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

RE: I notice that Ubuntu

Quote:
I notice that Ubuntu has the 270-beta in one of the repos.

which repo? i can't find this new 270 and my gtx 560 doesn't work without it ((

Mike Kubanek
Mike Kubanek
Joined: 5 Mar 08
Posts: 7
Credit: 25869651
RAC: 0

I just read the announcement

I just read the announcement that the new software will basically use the Nvidia GPU, I have a lot of host processors, but very few Nvidia GPU's, are my systems now obsolete, or have I misunderstood the message. I would like to continus to help out, but if all your using are the Nvidia, then I don't have the right configurations for this work any longer. I must be reading something wrong, others would have mentioned this as well?
Mike

Mike Kubanek

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686056996
RAC: 576866

RE: I just read the

Quote:
I just read the announcement that the new software will basically use the Nvidia GPU, I have a lot of host processors, but very few Nvidia GPU's, are my systems now obsolete, or have I misunderstood the message. I would like to continus to help out, but if all your using are the Nvidia, then I don't have the right configurations for this work any longer. I must be reading something wrong, others would have mentioned this as well?
Mike


Don't worry, both searches (S5GC1HF and BRP3) continue to support non-GPU hosts under Linux, Windows and Mac OS X.

CU
HB

Ver Greeneyes
Ver Greeneyes
Joined: 26 Mar 09
Posts: 140
Credit: 9562235
RAC: 0

On the other hand, using a

On the other hand, using a GPU greatly speeds up the Binary Radio Pulsar Search WUs - on my middling GeForce 250 GTS, the completion time drops to ~90 minutes compared to over 15 hours for my Q9450 @ 3.2 GHz. Of course, the CPU can do 4 WUs at the same time, but even then it's a big difference.

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

RE: I'm not sure whether

Quote:
I'm not sure whether this beta already contains the fix for the driver problem. I'll check.
...
This way, those who are brave enough to use a beta driver could already beta-test the Linux app in the state it will be for the new official driver.
Heinz

i'm ready to test. i have at least two gtx 560 and its performance in E@H greatly depends on main CPU speed =( this is sad, i can't run GPUs at full throttle.

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

RE: RE: If ever will be

Quote:
Quote:
If ever will be possible to crunch on ATI GPU? if believed to specs it has much more calculating power.

FYI...

thank you, but it is S@H related, not E@H.
what about E@H on ATI?

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: RE: RE: If ever

Quote:
Quote:
Quote:
If ever will be possible to crunch on ATI GPU? if believed to specs it has much more calculating power.

FYI...

thank you, but it is S@H related, not E@H.
what about E@H on ATI?

Sorry, I meant to quote this posting of mine...

 

Einstein@Home Project

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: and what's wrong with

Quote:
and what's wrong with mac? there is still no gpu app for it.

We see different issues on the Mac. We and NVIDIA still have to investigate this further...

Thanks for your patience,

Oliver

 

Einstein@Home Project

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: I think it's wise to

Quote:

I think it's wise to wait until the current cross-validation issues for the CUDA app are completely solved before introducing another platform.

As for the beta driver: Hmm... I'm not sure whether this beta already contains the fix for the driver problem. I'll check.

If so....hmm....maybe it would be a nice idea to release an app that could be triggered to disable the driver-bug workaround, e.g. by creating a special named file in the BOINC E@H project directory as was done for the GW app to enable some debugging features iirc. This way, those who are brave enough to use a beta driver could already beta-test the Linux app in the state it will be for the new official driver.

Oliver, what do you think?

CS
Heinz

I'll check the driver. If all works out well we could release an unofficial/beta version of our Linux app that doesn't contain the existing workaround and thus would use as less of a CPU core as possible.

Cheers,
Oliver

 

Einstein@Home Project

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

Update: I checked the

Update: I checked the 260.19.36 as well as the 270.18 beta driver. While the first doesn't fix the issue (as expected) the latter does. Well, at least in a first test run that still needs to finish... If all turns out fine we might publish an unofficial release of our Linux 32-bit app that relies on this new driver. You are free to install and run the new driver/app combo manually as you please (using an appropriate app_info.xml file).

Stay tuned for more...

Cheers,
Oliver

 

Einstein@Home Project

Markus
Markus
Joined: 31 May 09
Posts: 1
Credit: 7509
RAC: 0

and wat s happend if you use

and wat s happend if you use a ATI Radeon?

Michael Karlinsky
Michael Karlinsky
Joined: 22 Jan 05
Posts: 888
Credit: 23502182
RAC: 0

RE: and wat s happend if

Quote:
and wat s happend if you use a ATI Radeon?

Only NVIDIA is supported here ATM.

Michael

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

RE: If all turns out fine

Quote:
If all turns out fine we might publish an unofficial release of our Linux 32-bit app

That's great! but shall it run on 64 bit linux?

Oliver Behnke
Oliver Behnke
Moderator
Administrator
Joined: 4 Sep 07
Posts: 942
Credit: 25166626
RAC: 0

RE: RE: If all turns out

Quote:
Quote:
If all turns out fine we might publish an unofficial release of our Linux 32-bit app

That's great! but shall it run on 64 bit linux?

Sure, just install the usual 32-bit compatibility libs (should come in a (meta-)package with your distribution).

Oliver

 

Einstein@Home Project

SilentRAIDg
SilentRAIDg
Joined: 23 Nov 10
Posts: 1
Credit: 9885739
RAC: 0

I have to Disable my SLI

I have to Disable my SLI feature otherwise my computer lags and sticks when working with any desktop Application. My specs are :

EATX Tyan S2915-E Thunder Motherboard
2x Opteron 2352 Socket F Quadcore CPUs
8Gb Micron RAM
2x GeForce 9600GT SSC GPUs
Win7 Ultimate x64

Any suggestions?

Bruce Allen
Bruce Allen
Moderator
Joined: 15 Oct 04
Posts: 1119
Credit: 172127663
RAC: 0

RE: what about E@H on

Quote:
what about E@H on ATI?

For ATI GPU cards, we are working on an OpenCL version of the Einstein@Home Radio Pulsar Search application. I am optimistic that this will be available later in 2011, but substantial work is still needed to complete an efficient OpenCL port.

Director, Einstein@Home

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

we urgently need for a new

we urgently need for a new linux version (even beta) without bug to open a whole potential of our gpus =)

telegd
telegd
Joined: 17 Apr 07
Posts: 91
Credit: 10212522
RAC: 0

RE: RE: I notice that

Quote:
Quote:
I notice that Ubuntu has the 270-beta in one of the repos.

which repo? i can't find this new 270 and my gtx 560 doesn't work without it ((


It is in the x-swat ppa found here:
https://launchpad.net/~ubuntu-x-swat/+archive/x-updates

Caution - I have not tested it myself. I am very careful about upgrading my Nvidia drivers unless necessary. Even more so about beta versions.

It is great news that the 270 series might fix our problem, though. I look forward to hearing more!

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686056996
RAC: 576866

Hi! Experienced Linux

Hi!

Experienced Linux users can look here for new NVIDIA drivers:

(64 bit)
ftp://download.nvidia.com/XFree86/Linux-x86_64/

(32 bit)
ftp://download.nvidia.com/XFree86/Linux-x86/

where you will also find the 270 beta driver.

Those installers are not distribution specific and the installer might need to actually compile the kernel module by itself, so you will need a development environment (C compiler etc) installed, see the README. So if you can get a driver from the package manager of your distribution ==> get it from there :-). If you want to try out the latest beta, you might want to try the NVIDIA site.

I tried this on OpenSuse 11.2 and it looks promising so far. Knock on wood.

CU
HB

astrocrab
astrocrab
Joined: 28 Jan 08
Posts: 208
Credit: 429202534
RAC: 0

on ubuntu 10.10 we have this

on ubuntu 10.10 we have this beta for several weeks already )
all we need now is new linux cuda app without 260 bug ))

telegd
telegd
Joined: 17 Apr 07
Posts: 91
Credit: 10212522
RAC: 0

RE: I tried this on

Quote:
I tried this on OpenSuse 11.2 and it looks promising so far. Knock on wood.


Me too. I finally installed the 270.18 beta on Kubuntu (Mint) 10.04 and it seems to behaving well on the 1.06 BRPS Work-Units.

It does not seem to have fixed all of the cross-validation problems, though:
http://einsteinathome.org/workunit/93182449

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686056996
RAC: 576866

RE: RE: I tried this on

Quote:
Quote:
I tried this on OpenSuse 11.2 and it looks promising so far. Knock on wood.

Me too. I finally installed the 270.18 beta on Kubuntu (Mint) 10.04 and it seems to behaving well on the 1.06 BRPS Work-Units.

It does not seem to have fixed all of the cross-validation problems, though:
http://einsteinathome.org/workunit/93182449

The cross validation problem is not a driver issue, it will be fixed with the next app version. There's good progress on this problem, but that's a different topic.

As for the 270 driver: It runs flawlessly on my 16 bit Suse 11.2 box, but I got hanging CUDA apps since updating the driver on my 64 bit Linux box.

Does anybody see the same on a 64 bit Linux w/ 270 driver? I would not be surprised to see bugs like this in an early beta driver, so everybody giving this driver a try obviously should monitor the host closely.

CU
HB

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 556

RE: The cross validation

Quote:


The cross validation problem is not a driver issue, it will be fixed with the next app version. There's good progress on this problem, but that's a different topic.

As for the 270 driver: It runs flawlessly on my 16 bit Suse 11.2 box, but I got hanging CUDA apps since updating the driver on my 64 bit Linux box.

Does anybody see the same on a 64 bit Linux w/ 270 driver? I would not be surprised to see bugs like this in an early beta driver, so everybody giving this driver a try obviously should monitor the host closely.

CU
HB

I built driver 270.18 for my 64-bit Linux box yesterday. So far the CUDA app has been running stable. I'll keep an eye on it to see if the CUDA hang shows up on my system.

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686056996
RAC: 576866

Thanks! I got the

Thanks!

I got the impression that tasks hang only if started with run_client in the background without an actice X desktop, bute rarely if ever stall when started with run_manager and BOINC manager open. Just a first impression, tho.

CU

HB