the following is to give you a bit more background on the new binary radio pulsar search "BRP6" aka "PMPS XT" is about. The data your GPUs will be analyzing are archival observations from the Parkes Telescope in Australia, from the very successful Parkes Multi-beam Pulsar Survey (PMPS).
The very same data set has been previously analyzed using Einstein@Home in the BRP3 (PMPS) , which led to the discovery of 24 new radio pulsars, including quite a few interesting ones. You can read up on all the details and discoveries in the publication "Einstein@Home Discovery of 24 Pulsars in the Parkes Multi-beam Pulsar Survey", which is available for free on the arXiv.
When we planned and conducted this first analysis we where limited by the available computing power on Einstein@Home, which at that time forced us to limit our search to pulsars spinning with at most 130 Hz. Searching for faster spinning pulsars in binaries requires more templates, and the dependence on the spin frequency is pretty steep (f^3 for the experts). In the concluding paragraphs of the publication we speculated on what kind of searches we might be able to do in the future, assuming the usual (Moore's law) growth in computing power over time. Extrapolating from the computing power and apps back the we estimated that within a decade we would be able to re-do our analysis and search up to 250 Hz.
Here we are, a bit more than four years after the start of the BRP3 PMPS search. Improvements in the Nvidia BRP GPU apps and the development of an ATI/AMD GPU app allow us to already now extend our search range up to pulsar spin frequencies of 300 Hz. And that is precisely what BRP6 will do. With your help we will re-analyze the PMPS data and look for faster spinning pulsars in tight binary systems. Scientifically speaking, this is very interesting territority: fast-spinning (millisecond) pulsars in short-orbital-period binaries are an extremely exciting class of astronomical objects. One can do precise tests of general relativity with them, study stellar evolution, and get a more complete picture of the pulsar population in our Galaxy.
The PMPS data set still is quite large with over 41,000 individual observations. At the current computing power it would take about three years to conduct this very thorough analysis that nobody has ever done before. During this time, however, if the available computing power keeps growing as expected, the actual time to completion will be less.
Thank you for your support, keep on crunching,
Benjamin
...Improvements in the Nvidia BRP GPU apps and the development of an ATI/AMD GPU app allow us to already now extend our search range up to pulsar spin frequencies of 300 Hz...
Imagine what we could achieve with applications utilizing newer versions of CUDA, OpenCL or AVX.
Hi Ben ! This is all great news. May we find yet more rare birds .... :-)
Once upon a time I meant to ask about that PMPS survey as described in that 2013 E@H 24 pulsar paper ( page 2, section 2. PARKES MULTI-BEAM PULSAR SURVEY AND PREVIOUS ANALYSES ):
Quote:
.... with a total of 3190 telescope pointings (Manchester et al. 2001). Each pointing is comprised of 13 dual-polarization beams. These are arranged in two hexagonal rings, each containing six beams around a central beam, creating a ‘Star-of-David’ pattern (Staveley-Smith et al. 1996). Each observation covers a radio bandwidth of 288 MHz, which is observed in a filterbank of 96 channels each with 3MHz width, centered on 1374 MHz. The sampling time is 250 us and the filterbank data have a dynamic range of 1 bit per sample. Each beam has an integration time of 2097.152 s, with 2^23 time samples. This yields a file size of 100.7MB per beam and a total data volume of 4.1 TB for all filterbank and associated header files.
... I've pretty well got all that except the red highlighted bit. So that's basically an on/off switch, so to speak, for each 3MHz wide channel ? Have I read that right ? It seems rather coarse grained or rather more likely I'm missing something important here.
Cheers, Mike.
( edit ) I guess that relates also to the later comment :
Quote:
.... we convert the survey data in filterbank format from 1 bit dynamic range per sample into a file format with 8 bits per sample using ...
which I also don't follow. I guess I am thinking that for each given 250 us interval a certain frequency bin collects some number of photons ( of the energy corresponding to the frequency range in the bin ) which is then either above or below some threshold and then assigned a 0 or a 1 accordingly. That can't be right, can it ?
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
... I've pretty well got all that except the red highlighted bit. So that's basically an on/off switch, so to speak, for each 3MHz wide channel ? Have I read that right ? It seems rather coarse grained or rather more likely I'm missing something important here.
you have understood this correctly. In the PMPS data taken in the late 1990s, the data were encoded with 1 bit per sample. So at each timestamp (every 250 microseconds), the filterbank computes a radio spectrum of 96 channels covering 288 MHz of bandwidth. The values of that spectrum are stored in 1-bit format, i.e., an on-off switch. As far as I know that did a lot of testing and modelling beforehand and it turns out that you lose only a bit of SNR when encoding data in 1-bit format. Also, it makes your analysis more robust against very loud short-time outliers (like radio frequency interference).
You should also keep in mind, that for the actual pulsar searching you will de-disperse the data and sum the values in 96 channels, effectively averaging the 1-bit samples. At this point your resulting de-dispersed (time series) data will not be stored with 1-bit samples!
Regarding your other question about the 8-bit conversion issue: The quote from the paper is correct. The conversion only has to happen for technical reasons. The telescope data we feed into the pipeline still has 1 bit per sample, but the piece of code we used for de-dispersion from the standard PRESTO pulsar processing software suite needs 8 bit per sample at least. This is why we simply re-write the 1-bit-per-sample data into 8-bit-per-sample data. This step does not add any resolution or information.
Interesting, so if I understand correctly, it is plain 1-bit sampling (similar to PCM audio) and not a form of PWM (similar to DSD audio) I thought so far?
Interesting, so if I understand correctly, it is plain 1-bit sampling (similar to PCM audio) and not a form of PWM (similar to DSD audio) I thought so far?
yup, this is correct. The filterbank generates radio spectra for each time sample. The frequency bins of the radio spectrum contain the intensity (the two orthogonal polarizations squared and summed), and are 1-bit encoded when digitized.
Imagine what we could achieve with applications utilizing newer versions of CUDA, OpenCL or AVX.
AVX: That would be useful only for CPU apps, which we don't plan to have for BRP6 in the foreseeable future.
Newer versions of CUDA: Yes, that's on our list of TODOs for the near future, in fact the OSX-CUDA version we are now using for BRP6 is already CUDA 5.5.
Newer versions of OpenCL: not sure what to gain from that, but we might want to try a different FFT lib.
I also have one or two ideas for further optimizations that would reduce the amount of data transferred over the PCIe bus. As this BRP6 run will last for quite a while, I'm willing to invest some time again to make this more efficient.
AVX: That would be useful only for CPU apps, which we don't plan to have for BRP6 in the foreseeable future.
I thought not only about BRP6, but also about other applications/subprojects.
Quote:
Newer versions of CUDA: Yes, that's on our list of TODOs for the near future, in fact the OSX-CUDA version we are now using for BRP6 is already CUDA 5.5.
This is good news.
Quote:
Newer versions of OpenCL: not sure what to gain from that, but we might want to try a different FFT lib.
I was thinking about using shared memory in APUs, etc.
Quote:
I also have one or two ideas for further optimizations that would reduce the amount of data transferred over the PCIe bus. As this BRP6 run will last for quite a while, I'm willing to invest some time again to make this more efficient.
Hi all, the following is
)
Hi all,
the following is to give you a bit more background on the new binary radio pulsar search "BRP6" aka "PMPS XT" is about. The data your GPUs will be analyzing are archival observations from the Parkes Telescope in Australia, from the very successful Parkes Multi-beam Pulsar Survey (PMPS).
The very same data set has been previously analyzed using Einstein@Home in the BRP3 (PMPS) , which led to the discovery of 24 new radio pulsars, including quite a few interesting ones. You can read up on all the details and discoveries in the publication "Einstein@Home Discovery of 24 Pulsars in the Parkes Multi-beam Pulsar Survey", which is available for free on the arXiv.
When we planned and conducted this first analysis we where limited by the available computing power on Einstein@Home, which at that time forced us to limit our search to pulsars spinning with at most 130 Hz. Searching for faster spinning pulsars in binaries requires more templates, and the dependence on the spin frequency is pretty steep (f^3 for the experts). In the concluding paragraphs of the publication we speculated on what kind of searches we might be able to do in the future, assuming the usual (Moore's law) growth in computing power over time. Extrapolating from the computing power and apps back the we estimated that within a decade we would be able to re-do our analysis and search up to 250 Hz.
Here we are, a bit more than four years after the start of the BRP3 PMPS search. Improvements in the Nvidia BRP GPU apps and the development of an ATI/AMD GPU app allow us to already now extend our search range up to pulsar spin frequencies of 300 Hz. And that is precisely what BRP6 will do. With your help we will re-analyze the PMPS data and look for faster spinning pulsars in tight binary systems. Scientifically speaking, this is very interesting territority: fast-spinning (millisecond) pulsars in short-orbital-period binaries are an extremely exciting class of astronomical objects. One can do precise tests of general relativity with them, study stellar evolution, and get a more complete picture of the pulsar population in our Galaxy.
The PMPS data set still is quite large with over 41,000 individual observations. At the current computing power it would take about three years to conduct this very thorough analysis that nobody has ever done before. During this time, however, if the available computing power keeps growing as expected, the actual time to completion will be less.
Thank you for your support, keep on crunching,
Benjamin
Einstein@Home Project
Thank you for your
)
Thank you for your comprehensive information.
Imagine what we could achieve with applications utilizing newer versions of CUDA, OpenCL or AVX.
I just put a new cruncher
)
I just put a new cruncher online and after reading the post by Benjamin I'm ordering parts for another new one.
This is exciting information, keep it coming when you can. It is very much appreciated.
Phil
I thought I was wrong once, but I was mistaken.
Is this new application
)
Is this new application limited to GPUs only? I would like to help out with my CPUs.
Hi Ben ! This is all great
)
Hi Ben ! This is all great news. May we find yet more rare birds .... :-)
Once upon a time I meant to ask about that PMPS survey as described in that 2013 E@H 24 pulsar paper ( page 2, section 2. PARKES MULTI-BEAM PULSAR SURVEY AND PREVIOUS ANALYSES ):
... I've pretty well got all that except the red highlighted bit. So that's basically an on/off switch, so to speak, for each 3MHz wide channel ? Have I read that right ? It seems rather coarse grained or rather more likely I'm missing something important here.
Cheers, Mike.
( edit ) I guess that relates also to the later comment :
which I also don't follow. I guess I am thinking that for each given 250 us interval a certain frequency bin collects some number of photons ( of the energy corresponding to the frequency range in the bin ) which is then either above or below some threshold and then assigned a 0 or a 1 accordingly. That can't be right, can it ?
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
Hi Mike, RE: ...
)
Hi Mike,
you have understood this correctly. In the PMPS data taken in the late 1990s, the data were encoded with 1 bit per sample. So at each timestamp (every 250 microseconds), the filterbank computes a radio spectrum of 96 channels covering 288 MHz of bandwidth. The values of that spectrum are stored in 1-bit format, i.e., an on-off switch. As far as I know that did a lot of testing and modelling beforehand and it turns out that you lose only a bit of SNR when encoding data in 1-bit format. Also, it makes your analysis more robust against very loud short-time outliers (like radio frequency interference).
You should also keep in mind, that for the actual pulsar searching you will de-disperse the data and sum the values in 96 channels, effectively averaging the 1-bit samples. At this point your resulting de-dispersed (time series) data will not be stored with 1-bit samples!
Regarding your other question about the 8-bit conversion issue: The quote from the paper is correct. The conversion only has to happen for technical reasons. The telescope data we feed into the pipeline still has 1 bit per sample, but the piece of code we used for de-dispersion from the standard PRESTO pulsar processing software suite needs 8 bit per sample at least. This is why we simply re-write the 1-bit-per-sample data into 8-bit-per-sample data. This step does not add any resolution or information.
Cheers,
Benjamin
edit: added links
Einstein@Home Project
Interesting, so if I
)
Interesting, so if I understand correctly, it is plain 1-bit sampling (similar to PCM audio) and not a form of PWM (similar to DSD audio) I thought so far?
Hi
)
Hi Sebastian,
yup, this is correct. The filterbank generates radio spectra for each time sample. The frequency bins of the radio spectrum contain the intensity (the two orthogonal polarizations squared and summed), and are 1-bit encoded when digitized.
Cheers,
Benjamin
Einstein@Home Project
RE: Imagine what we could
)
AVX: That would be useful only for CPU apps, which we don't plan to have for BRP6 in the foreseeable future.
Newer versions of CUDA: Yes, that's on our list of TODOs for the near future, in fact the OSX-CUDA version we are now using for BRP6 is already CUDA 5.5.
Newer versions of OpenCL: not sure what to gain from that, but we might want to try a different FFT lib.
I also have one or two ideas for further optimizations that would reduce the amount of data transferred over the PCIe bus. As this BRP6 run will last for quite a while, I'm willing to invest some time again to make this more efficient.
Cheers
HB
RE: AVX: That would be
)
I thought not only about BRP6, but also about other applications/subprojects.
This is good news.
I was thinking about using shared memory in APUs, etc.
This is even better news.