Is it actually possible to see some of the data that we are processing?
The original signal data comes from the gravity wave detector, or the Arecibo antenna.
I am going to assume that you mean the results that are produced from the original signal data. (the stuff that gets sent back to the server from your PC).
I am probably not the best person to answer this question, but I will provide you a link to a thread that was trying to provide a Screen Saver that would show the actual results that a single WU was creating.
You have to be familiar with the current Einstein@Home screen saver. The improved screen saver was never completed, but the graphics can help understand what is being calculated. The colors are signal intensity.
You will need to copy the link and past it into your browsers address bar. The link is not clickable.
Actually, I am interested in the raw data coming from the gravity wave detector.
Have you had a chance to skim through any of the Detector Watch threads? There are many kinds of 'raw' data that are taken into account. I think the raw output of the interferometer itself is the power (in Watts) from a photodiode at one of the ports, and that by itself doesn't mean a whole lot – what would you do with that data? It would be interesting to sift it for anomalies in a general way and see how they compare to data that's been properly 'crunched', and also compare them to any other available concurrent datasets from nature (like from here) just to see if there are any strong correlations with anything else. Am guessing that raw LIGO data is available in some format – ah-ha – try here. Does that help?
Actually the input data that E@H processes is NOT the raw data from LIGOs, GW channel, it's already preprocessed. But to understand why and how, let me explain this in some detail:
The type of signal that we are looking for here is very weak, but continuous: it is sent out by the source all day long, for years, even centuries, in a rather predictable way. The idea behind E@H is to catch such a signal by looking at a very long stretch of data, somehow integrating (in the sense of combining) the signal over the whole observation time and make it thereby visible even though it's very weak and buried in a lot of noise. A bit like making a long time exposure (or "stacking" several exposures) in conventional astrophotography, perhaps.
Let's say we want 3 months of net observation time. That's about 7.7 million seconds. At a sampling frequency of 16 kHz, we are talking about raw data from each detector in the three digit GB range!! And because it's almost pure noise, it doesn't even compress well :-).
You can't ship out Gigabytes of data to the BOINC volunteers! And it's impossible to divide up that data by time of recording (say, send 5 minutes worth of data to each host) because that defies the whole purpose: combine the data over the whole observation period by the hosts. There must be other ways to split up the data.
Some audio compression methods like MP3 use the following trick: they take the original recording and represent it in another way: as the sum of sine waves of different frequencies. Then they decide to drop some waves(=frequencies) because the human ear/brain just won't notice them missing and voila, the data is reduced.
For the processing of LIGO data for E@H, a very similar trick is used, but not to compress the data, instead it makes it easier to handle and analyze it (note: this preprocessing happens on the servers, not the E@H hosts!).
At first, the raw data is sliced into segments of half an hour each. Now each segment is broken down by a method called FFT (fast Fourier transform) into simple sine waves so that the sum of the sine waves is the original signal. .
What have we gained? We can now conveniently split the data along the frequency axis, and deliver data for the full observation period to the clients. Have you ever wondered what those files with names like "h1_0123.05_S5R4" that are downloaded by the app are good for? They are each about 4 MB in size, and they contain detector output data for the whole observation period (!), but only for a small frequency band (in this example, h1_0123.05_S5R4 contains data from the Hanford observatory, for frequencies in the range 123.05 ... 123.10 Hz ).
Think of the input files as a two dimensional table: The columns represent observation time, one column for every 30 minute interval.
The rows represent frequency: every row contains the amplitudes for a certain, tiny frequency "bin" (like the equalizer display on your stereo displays one bar for every "frequency bin" ).
So would you be able to see a GW signal if you plotted this 2 D data? Or would we hear something by playing it through a speaker? No. There are two problems:
1) The signal will be so very weak that it needs some clever signal processing by the E@H client to detect it beneath all the noise.
2) A GW source that emits a signal at a given frequency will not generate a constant signal with the same frequency in LIGO! Instead the signal will be Doppler shifted because of the Earth's motion around the Sun and its daily rotation. So you cannot just look at every single frequency bin and do clever statistics to uncover a weak signal: the signal you are looking for will be hidden in several bins, but which bins depends on the source position in the sky..which you don't know.
That's were the E@H client steps in.
I hope I didn't misrepresent anything too much (not being a physicist).
I remember reading about Wavelet analysis once - the paper mentioned that where harmonic analysis is insensitive to time-specific (rather than frequency-specific) features, wavelets can capture both elements. If the noise, or even the GWs themselves aren't purely harmonic, do you think a wavelet analysis could do a better job than an FFT? In the paper, they gave the example of filtering out weak earthquake signals from many months of noisy background signal - the result was impressive. I think it's also a matter of how well you understand the signal though, as wavelets allow you to choose any shape of wave(let) you want. Using the results is still based on binning, so you'd have to know what to try and filter out! (unfortunately, the technical details of the paper went over my head - even though it was called 'Wavelet theory for kids'! Some kids, those..)
I remember reading about Wavelet analysis once - the paper mentioned that where harmonic analysis is insensitive to time-specific (rather than frequency-specific) features, wavelets can capture both elements. If the noise, or even the GWs themselves aren't purely harmonic, do you think a wavelet analysis could do a better job than an FFT? In the paper, they gave the example of filtering out weak earthquake signals from many months of noisy background signal - the result was impressive. I think it's also a matter of how well you understand the signal though, as wavelets allow you to choose any shape of wave(let) you want. Using the results is still based on binning, so you'd have to know what to try and filter out! (unfortunately, the technical details of the paper went over my head - even though it was called 'Wavelet theory for kids'! Some kids, those..)
Indeed, you'll find several papers discussing using wavelet analysis to find gravitational waves, but for transient or burst signals. E@H is looking for continuous waves from rotating neutron stars which are thought to be extremely regular. Therefore a Fourier transform of 30 minute segments (actually what is used by E@H is called an SFT, see http://en.wikipedia.org/wiki/Short-time_Fourier_transform) should work just fine without adding additional computational complexity.
Indeed, you'll find several papers discussing using wavelet analysis to find gravitational waves, but for transient or burst signals. E@H is looking for continuous waves from rotating neutron stars which are thought to be extremely regular. Therefore a Fourier transform of 30 minute segments (actually what is used by E@H is called an SFT, see http://en.wikipedia.org/wiki/Short-time_Fourier_transform) should work just fine without adding additional computational complexity.
Thanks for the answer :) So we can filter out peaks in the noise because we know they won't be part of the regular wave data anyway, and the rest is covered by harmonic analysis - makes sense to me! Now let's hope we find some ;)
On the 15 January issue of New Scientist there is an article "Our world might be a giant hologram", which I don't link because I don't understand the New Scientist's policy of allowing or not the linking of its articles, which says that the GEO600 interferometer has found a noise in its data which might be explained by a granularity of space-time. I don't even try to resume the article's content, just go and read it by googling "hologram", page 2. My question is: has LIGO found a similar noise, or is it in a frequency band not recorded by its interferometers?
Tullio
Could this be last year's January edition? This debate was pretty hot last year but I haven't heart much of it since then. The initial claim was that because of some special features of the GEO600 optics, the effect could only be seen in GEO (if at all).
Could this be last year's January edition? This debate was pretty hot last year but I haven't heart much of it since then. The initial claim was that because of some special features of the GEO600 optics, the effect could only be seen in GEO (if at all).
CU
HB
Of course you are right. It was January 2009.
Tullio
What we process
)
The original signal data comes from the gravity wave detector, or the Arecibo antenna.
I am going to assume that you mean the results that are produced from the original signal data. (the stuff that gets sent back to the server from your PC).
I am probably not the best person to answer this question, but I will provide you a link to a thread that was trying to provide a Screen Saver that would show the actual results that a single WU was creating.
You have to be familiar with the current Einstein@Home screen saver. The improved screen saver was never completed, but the graphics can help understand what is being calculated. The colors are signal intensity.
You will need to copy the link and past it into your browsers address bar. The link is not clickable.
THREAD:Visualization of S5R2 / S5R3 difference
http://einsteinathome.org/node/193259&nowrap=true
--- 1st message in thread------
Hi!
I was playing around with a tool called "Topcat", visualizing Einstein@Home result files (the stuff that gets sent back to the server from your PC).
The first picture shows a visualization of an actual S5R2 result file (h1_0281.20_S5R2__65_S5R2c_4_0 to be exact).
---- end of 1st msg -----
Hope this gives you a start.
Actually, I am interested in
)
Actually, I am interested in the raw data coming from the gravity wave detector.
RE: Actually, I am
)
Have you had a chance to skim through any of the Detector Watch threads? There are many kinds of 'raw' data that are taken into account. I think the raw output of the interferometer itself is the power (in Watts) from a photodiode at one of the ports, and that by itself doesn't mean a whole lot – what would you do with that data? It would be interesting to sift it for anomalies in a general way and see how they compare to data that's been properly 'crunched', and also compare them to any other available concurrent datasets from nature (like from here) just to see if there are any strong correlations with anything else. Am guessing that raw LIGO data is available in some format – ah-ha – try here. Does that help?
Actually the input data that
)
Actually the input data that E@H processes is NOT the raw data from LIGOs, GW channel, it's already preprocessed. But to understand why and how, let me explain this in some detail:
The type of signal that we are looking for here is very weak, but continuous: it is sent out by the source all day long, for years, even centuries, in a rather predictable way. The idea behind E@H is to catch such a signal by looking at a very long stretch of data, somehow integrating (in the sense of combining) the signal over the whole observation time and make it thereby visible even though it's very weak and buried in a lot of noise. A bit like making a long time exposure (or "stacking" several exposures) in conventional astrophotography, perhaps.
Let's say we want 3 months of net observation time. That's about 7.7 million seconds. At a sampling frequency of 16 kHz, we are talking about raw data from each detector in the three digit GB range!! And because it's almost pure noise, it doesn't even compress well :-).
You can't ship out Gigabytes of data to the BOINC volunteers! And it's impossible to divide up that data by time of recording (say, send 5 minutes worth of data to each host) because that defies the whole purpose: combine the data over the whole observation period by the hosts. There must be other ways to split up the data.
Some audio compression methods like MP3 use the following trick: they take the original recording and represent it in another way: as the sum of sine waves of different frequencies. Then they decide to drop some waves(=frequencies) because the human ear/brain just won't notice them missing and voila, the data is reduced.
For the processing of LIGO data for E@H, a very similar trick is used, but not to compress the data, instead it makes it easier to handle and analyze it (note: this preprocessing happens on the servers, not the E@H hosts!).
At first, the raw data is sliced into segments of half an hour each. Now each segment is broken down by a method called FFT (fast Fourier transform) into simple sine waves so that the sum of the sine waves is the original signal. .
What have we gained? We can now conveniently split the data along the frequency axis, and deliver data for the full observation period to the clients. Have you ever wondered what those files with names like "h1_0123.05_S5R4" that are downloaded by the app are good for? They are each about 4 MB in size, and they contain detector output data for the whole observation period (!), but only for a small frequency band (in this example, h1_0123.05_S5R4 contains data from the Hanford observatory, for frequencies in the range 123.05 ... 123.10 Hz ).
Think of the input files as a two dimensional table: The columns represent observation time, one column for every 30 minute interval.
The rows represent frequency: every row contains the amplitudes for a certain, tiny frequency "bin" (like the equalizer display on your stereo displays one bar for every "frequency bin" ).
So would you be able to see a GW signal if you plotted this 2 D data? Or would we hear something by playing it through a speaker? No. There are two problems:
1) The signal will be so very weak that it needs some clever signal processing by the E@H client to detect it beneath all the noise.
2) A GW source that emits a signal at a given frequency will not generate a constant signal with the same frequency in LIGO! Instead the signal will be Doppler shifted because of the Earth's motion around the Sun and its daily rotation. So you cannot just look at every single frequency bin and do clever statistics to uncover a weak signal: the signal you are looking for will be hidden in several bins, but which bins depends on the source position in the sky..which you don't know.
That's were the E@H client steps in.
I hope I didn't misrepresent anything too much (not being a physicist).
CU
Bikeman
I remember reading about
)
I remember reading about Wavelet analysis once - the paper mentioned that where harmonic analysis is insensitive to time-specific (rather than frequency-specific) features, wavelets can capture both elements. If the noise, or even the GWs themselves aren't purely harmonic, do you think a wavelet analysis could do a better job than an FFT? In the paper, they gave the example of filtering out weak earthquake signals from many months of noisy background signal - the result was impressive. I think it's also a matter of how well you understand the signal though, as wavelets allow you to choose any shape of wave(let) you want. Using the results is still based on binning, so you'd have to know what to try and filter out! (unfortunately, the technical details of the paper went over my head - even though it was called 'Wavelet theory for kids'! Some kids, those..)
RE: I remember reading
)
Indeed, you'll find several papers discussing using wavelet analysis to find gravitational waves, but for transient or burst signals. E@H is looking for continuous waves from rotating neutron stars which are thought to be extremely regular. Therefore a Fourier transform of 30 minute segments (actually what is used by E@H is called an SFT, see http://en.wikipedia.org/wiki/Short-time_Fourier_transform) should work just fine without adding additional computational complexity.
CU
HB
RE: Indeed, you'll find
)
Thanks for the answer :) So we can filter out peaks in the noise because we know they won't be part of the regular wave data anyway, and the rest is covered by harmonic analysis - makes sense to me! Now let's hope we find some ;)
On the 15 January issue of
)
On the 15 January issue of New Scientist there is an article "Our world might be a giant hologram", which I don't link because I don't understand the New Scientist's policy of allowing or not the linking of its articles, which says that the GEO600 interferometer has found a noise in its data which might be explained by a granularity of space-time. I don't even try to resume the article's content, just go and read it by googling "hologram", page 2. My question is: has LIGO found a similar noise, or is it in a frequency band not recorded by its interferometers?
Tullio
Could this be last year's
)
Could this be last year's January edition? This debate was pretty hot last year but I haven't heart much of it since then. The initial claim was that because of some special features of the GEO600 optics, the effect could only be seen in GEO (if at all).
CU
HB
RE: Could this be last
)
Of course you are right. It was January 2009.
Tullio