Out of Arecibo data

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022
Topic 195487

As you may have noticed, (and was anticipated here) task generation for ABP2 has stopped, and all the generated ABP2 tasks were sent out. Actually the data lasted about a week longer than I originally expected.

Parkes data has arrived @AEI, the new workunit generator and the setup is almost ready, but still needs some work and testing. I expect the project will be without Radio-Pulsar work for up to a week.

We are still looking into the remaining issues with a new generation of (CUDA) applications, it is not clear yet whether we'll launch the new App together with the new data or at a later time.

BM

BM

leks
leks
Joined: 21 Nov 07
Posts: 28
Credit: 483169031
RAC: 0

Out of Arecibo data

But we have a lot of gray dots in the "ABP Progress page, especially Center N-grid and Center S-grid

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022

Well, these graphs record

Well, these graphs record "finished" datasets (beams), i.e. where all tasks of this dataset have been uploaded, reported, validated and assimiated. Most of the tasks for these datasets have been sent out, but are still "in progress" (i.e. being computed in the next days).

In addition there are a few data sets which we couldn't process with our current workunit generator, e.g. because teh data was too noisy. We'll try to feed this data into the new workunit generator when it's ready, but I doubt that we'll find something in these datasets anyway.

BM

BM

tolafoph
tolafoph
Joined: 14 Sep 07
Posts: 122
Credit: 74659937
RAC: 0

RE: We are still looking

Quote:
We are still looking into the remaining issues with a new generation of (CUDA) applications, it is not clear yet whether we'll launch the new App together with the new data or at a later time.

Hi,

will there be a beta testing phase for the new App?

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022

RE: will there be a beta

Quote:
will there be a beta testing phase for the new App?

Not in the sense we previously had beta testing application versions here on Einstein@home.

Although the new app does essentially the same as the old (ABP2) one, it does some things a little differently, so that the results of the two apps don't validate against one another. This will be a completely new application, probably named "BRP3" (Binary Radio Pulsar search, as not limited to Arecibo data anymore), with both CPU and CUDA versions (and possibly OpenCL later).

BM

Edit: my earlier post was ambiguous: The new app will have both CPU and CUDA versions, however the current issues we have only with the CUDA versions of that app.

BM

Speedy
Speedy
Joined: 11 Aug 05
Posts: 39
Credit: 22006889
RAC: 521

When the new CUDA app comes

When the new CUDA app comes out will it's CPU usage be refined a little eg using 0.5 CPU or will CUDA require 1 CPU (core) too 1 GPU?

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022

RE: When the new CUDA app

Quote:
When the new CUDA app comes out will it's CPU usage be refined a little eg using 0.5 CPU or will CUDA require 1 CPU (core) too 1 GPU?

The next generation of CUDA Apps will do almost all computation on the GPU, depending on the power of your GPU and CPU it should take around 10% of a CPU core.

BM

BM

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109376556220
RAC: 35995803

How close are you now to

How close are you now to starting to distribute Parkes data and the new CPU app to crunch it?

Also, how long is the Parkes data likely to last? If either Arecibo or Parkes data is likely to last for a reasonable time, I might need to review my policy about not owning NVIDIA GPUs :-).

Cheers,
Gary.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022

I think that we're pretty

I think that we're pretty close to shipping out new work for ABP2 (Parkes and some Arecibo data we couldn't process with our old workunit generator). I just don't know how the tests went - I'll be back in my office tomorrow.

Parkes data should last for more than a year, if we (can) push the computing power of the RP search to 50% of the current E@H capability. Even longer if not.

As for the new CUDA App we're waiting for feedback / help / fix from NVidia. The problem is not in our code. It's something going wrong between the code that NVidias compiler (nvcc) produces and the CUDA driver that should use it.

BM

BM

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4264
Credit: 244920768
RAC: 17022

Update: The old ABP2 App

Update:

The old ABP2 App has a flaw in the harmonic summing code that costs a bit of sensitivity (~2%). Although this isn't severe for the old Arecibo data we already processed, we don't want to carry this over to the new data and pipeline. Thus we will launch a new application for the new data.

However with the new application there is more than one problem with the CUDA code. We are currently testing and discussing whether we will start the new App with only CPU versions, backport the new harmonic summing code to the previous (somewhat inefficient) ABP2 CUDA App, or maybe start with CUDA Apps for just one platform.

In any case shipping out new radio-pulsar search data will take longer than we expected. I hope hat we have at least something at the end of the week, though this will probably not the optimal and final solution.

BM

BM

Rechenkuenstler
Rechenkuenstler
Joined: 22 Aug 10
Posts: 138
Credit: 102567115
RAC: 0

RE: Update: The old ABP2

Quote:

Update:

The old ABP2 App has a flaw in the harmonic summing code that costs a bit of sensitivity (~2%). Although this isn't severe for the old Arecibo data we already processed, we don't want to carry this over to the new data and pipeline. Thus we will launch a new application for the new data.

However with the new application there is more than one problem with the CUDA code. We are currently testing and discussing whether we will start the new App with only CPU versions, backport the new harmonic summing code to the previous (somewhat inefficient) ABP2 CUDA App, or maybe start with CUDA Apps for just one platform.

In any case shipping out new radio-pulsar search data will take longer than we expected. I hope hat we have at least something at the end of the week, though this will probably not the optimal and final solution.

BM

Bernd,

we had a similar discussion in another thread, whether to roll out patches for every change, or to collect and roll out as release block. I think, it's the same with the Cuda problem. Please be patient and take care for the quality of the roll out versions. To hurry and do many things in parallel, just for rolling out fast, will not pay off. You will have bug fixes and retests (for several platforms and versions) and a lot of other things, that will prevent you from developing and rolling out the main application.

Make a release and roll out plan which indicates how and when you plan to start with which version, communicate this plan to us and focus on the next release and make it up and running. Then start working on the next release.

There is so much work to do (e.g. for GWHF), that we actually do not really need all ABP versions within the next two or three days. It's fine to start with one version, then come up with the next versions step by step. But we expect, that the applications, we are runing, are working correct and stable.

Lokking foreward to reading your oppinion

Regards

Bernhard

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.