Out of Arecibo data
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4307
Credit: 249624043
RAC: 34207
29 Nov 2010 8:48:12 UTC
Topic 195487
(moderation:
As you may have noticed, (and was anticipated here) task generation for ABP2 has stopped, and all the generated ABP2 tasks were sent out. Actually the data lasted about a week longer than I originally expected.
Parkes data has arrived @AEI, the new workunit generator and the setup is almost ready, but still needs some work and testing. I expect the project will be without Radio-Pulsar work for up to a week.
We are still looking into the remaining issues with a new generation of (CUDA) applications, it is not clear yet whether we'll launch the new App together with the new data or at a later time.
BM
BM
Out of Arecibo data
)
But we have a lot of gray dots in the "ABP Progress page, especially Center N-grid and Center S-grid
Well, these graphs record
)
Well, these graphs record "finished" datasets (beams), i.e. where all tasks of this dataset have been uploaded, reported, validated and assimiated. Most of the tasks for these datasets have been sent out, but are still "in progress" (i.e. being computed in the next days).
In addition there are a few data sets which we couldn't process with our current workunit generator, e.g. because teh data was too noisy. We'll try to feed this data into the new workunit generator when it's ready, but I doubt that we'll find something in these datasets anyway.
BM
BM
RE: We are still looking
)
Hi,
will there be a beta testing phase for the new App?
RE: will there be a beta
)
Not in the sense we previously had beta testing application versions here on Einstein@home.
Although the new app does essentially the same as the old (ABP2) one, it does some things a little differently, so that the results of the two apps don't validate against one another. This will be a completely new application, probably named "BRP3" (Binary Radio Pulsar search, as not limited to Arecibo data anymore), with both CPU and CUDA versions (and possibly OpenCL later).
BM
Edit: my earlier post was ambiguous: The new app will have both CPU and CUDA versions, however the current issues we have only with the CUDA versions of that app.
BM
When the new CUDA app comes
)
When the new CUDA app comes out will it's CPU usage be refined a little eg using 0.5 CPU or will CUDA require 1 CPU (core) too 1 GPU?
RE: When the new CUDA app
)
The next generation of CUDA Apps will do almost all computation on the GPU, depending on the power of your GPU and CPU it should take around 10% of a CPU core.
BM
BM
How close are you now to
)
How close are you now to starting to distribute Parkes data and the new CPU app to crunch it?
Also, how long is the Parkes data likely to last? If either Arecibo or Parkes data is likely to last for a reasonable time, I might need to review my policy about not owning NVIDIA GPUs :-).
Cheers,
Gary.
I think that we're pretty
)
I think that we're pretty close to shipping out new work for ABP2 (Parkes and some Arecibo data we couldn't process with our old workunit generator). I just don't know how the tests went - I'll be back in my office tomorrow.
Parkes data should last for more than a year, if we (can) push the computing power of the RP search to 50% of the current E@H capability. Even longer if not.
As for the new CUDA App we're waiting for feedback / help / fix from NVidia. The problem is not in our code. It's something going wrong between the code that NVidias compiler (nvcc) produces and the CUDA driver that should use it.
BM
BM
Update: The old ABP2 App
)
Update:
The old ABP2 App has a flaw in the harmonic summing code that costs a bit of sensitivity (~2%). Although this isn't severe for the old Arecibo data we already processed, we don't want to carry this over to the new data and pipeline. Thus we will launch a new application for the new data.
However with the new application there is more than one problem with the CUDA code. We are currently testing and discussing whether we will start the new App with only CPU versions, backport the new harmonic summing code to the previous (somewhat inefficient) ABP2 CUDA App, or maybe start with CUDA Apps for just one platform.
In any case shipping out new radio-pulsar search data will take longer than we expected. I hope hat we have at least something at the end of the week, though this will probably not the optimal and final solution.
BM
BM
RE: Update: The old ABP2
)
Bernd,
we had a similar discussion in another thread, whether to roll out patches for every change, or to collect and roll out as release block. I think, it's the same with the Cuda problem. Please be patient and take care for the quality of the roll out versions. To hurry and do many things in parallel, just for rolling out fast, will not pay off. You will have bug fixes and retests (for several platforms and versions) and a lot of other things, that will prevent you from developing and rolling out the main application.
Make a release and roll out plan which indicates how and when you plan to start with which version, communicate this plan to us and focus on the next release and make it up and running. Then start working on the next release.
There is so much work to do (e.g. for GWHF), that we actually do not really need all ABP versions within the next two or three days. It's fine to start with one version, then come up with the next versions step by step. But we expect, that the applications, we are runing, are working correct and stable.
Lokking foreward to reading your oppinion
Regards
Bernhard