Weave Analysis Software for Follow-Up Searches

Or use command line switches for each parameter of the CPU or GPU applications:

    - maximum memory
    - Number of CPU Threads
    - Workgroup/kernel size

and so on.

This works well for the optimized Seti applications. Optimum adaptations to your own hardware are therefore possible:

In my case for AMD/OpenCL WUs file:  mb_cmdline-8.22_windows_intel__opencl_ati5_SoG.txt

-high_perf -hp high_perf -period_iterations_num 1 -sbs 2048 -spike_fft_thresh 4096 -tune 1 60 1 4 -oclfft_tune_gr 256 -oclfft_tune_lr 16 -oclfft_tune_wg 256 -oclfft_tune_ls 512 -oclfft_tune_bn 60 -oclfft_tune_cw 60 -tt 1500

The high end of that memory usage would clobber all of my hosts.  My 3 desktops are all quad core i7's with 32, 32, and 18GB of ram and running 5x CPU and 3x GPU tasks when running an all E@H load. 

The first two both have 4x8gb of DDR3, so I couldn't put more ram in without having to buy expensive and slower 16gb dimms to swap out the 8s.  One is a pure crunch box and probably could take ~5.5gb/task without choking.  The other one is my daily driver though and a much larger share of its ram is in use day to day (my browsers are currently pigging down on 7gb, which isn't out of normal for my usage).  I'd probably top out at being able to support ~4gb/task here today (and presumably a fair amount less over the next 2-3 years before I replace it).

My third box is a decadish old relic, with 3x4 and 3x2gb dimms in it.  Apparently it can support 8gb dimms but due to its age buying more ram is out of the question.   In any event I'm considering standing it's CPU down, putting both 1070's in it and putting my AMD 560 in box 2.  At that point I'd be running 7 CPU tasks and down to ~4gb/cpu task again.

When I built the newer two boxes I really thought 32gb of ram would be enough to keep them running for 8-10 years with only GPU swaps needed.

With mainstream CPU core counts rising steadily the ram/core situation isn't likely to get any better, and potentially will get a lot worse.  Current mainstream CPUs are up to 8/16 cores with the rumor mill suggesting that Intel's next generation will inch up to 10/20 cores, while AMD leapfrogs to 16/32.  With 32 gb ddr4 dimms stupidly expensive and mainstream sockets still limited to 2 channels/4 dimms, those systems are effectively topped out at 64gb of ram.

The amount of memory used is mostly dependent on how the data is chunked up in creating the WUs; which I think would be a deal breaker since if I'm understanding it correctly the one parameter that could be adjusted without changing how the WUs are generated would require 2x as much time to halve ram usage.

OTOH the Weave search appears like it would be doable for a multi-threaded app and being able to have all your cores pounding away at the same WU would allow for much higher per WU ram usage.  The gotcha there is that it'd greatly expand the runtime disparity between computers because high end desktops have both significantly more and significantly faster cores than mainstream laptops.

Also of note is the science payback : as 57.9/50.8 ~ 1.14, with (1.14)³ ~ 1.48 then that's how you get an extra ~ 50% of sky volume out there being covered c/w current methods ( all else being equal etc ). Isn't that neat ? Plus the follow up of candidates is two orders faster. This is a great piece of work. I wish I understood it in detail ..... :-)))

Cheers, Mike.

( edit ) Could be a good time to go over what 'coherent' vs 'incoherent' means in our setting :

An assumed waveform is a template, that representing the strength of a gravitational wave ( spacetime strain ) varying with time. But a template is only one cycle of some wave and we are looking for, or assuming, that a waveform repeats in time. Hence the term continuous waves. Imagine being at a beach with some continuous waves coming onto shore. Suppose you were a wizard & could dictate their shape, snap your fingers and tada ! All the waves coming in all look the same, break the same etc.

Now keep a certain template ( wave shape ) but speed up or slow down how frequently the waves arrive. Same template, different frequencies. From one wave to the next you still see the same waveform go by.

Now just to be slightly more complicated : allow for some change in this arrival frequency while you are watching. Maybe back at the beach you'd find the interval b/w waves ( their period in time ) reaching shore is smaller than you had noted beforehand, hence the wave frequency has increased. Conversely if the interval were increasing as you watch then we'd say their frequency was decreasing. How quickly the frequency changes over time is the frequency derivative or 'f-dot'. If f-dot is positive then the frequency is increasing, if negative then the frequency is decreasing.

Next pretend that we are on a circular island, neither too 'big' nor too 'small', such that we might go to different parts of the shoreline. We could see different wave behaviours ( waveform, frequency, f-dot ) depending upon where we went. We might set up a system whereby we designate which way we are facing ie. what compass direction is the distant horizon as we look offshore at some spot. Northwards could be zero in this system, Eastwards could then be 90, Southwards is 180 etc in the usual fashion. So to fully describe an observation we have an angle as well as the aforementioned wave behaviour stuff.

But we are not on a circular island. We are on an ( oh so very close to ) spherical ball in space. We just need another angle to describe which way we are facing as the wave data comes in. That extra angle would be analogous to looking above or below the horizon if we were on that island.

In summary then we have : a waveform, it's frequency, the frequency derivative, with lastly those two angles to designate our observation direction. If I'd spent a day watching the beach at some spot, with a single wave type coming in, I could broadly summarise my wave-watching experience to you in a letter with just these pieces of information. When you read the letter you'd have a very good idea what sort of a day I'd had, wave-wise.

So much for definitions. Now what do we actually do at E@H ??? :-) LOL

In effect, please think of it in the following way : you have a record of what actually happened at the sea shore for some length of time. It is not some well defined wizard-induced train of identical waves. No way. In fact it's likely to :

(a) Have not much going on. Gravity is 'quiet'. It takes a zillion dollars to just make machines that can accurately listen to these very soft signals.

(b) At a first glance look chaotic. No discernible pattern at all. But there is one saving feature which turns out to be the lifeline for the whole wave-watching enterprise.

In the circumstances under which we listen the whole business is linear. Basically that means when adding two waves together, the height of the resultant wave is the sum of the heights of the two waves separately. Extend that to very many waves superposed upon one another. Maybe, just maybe, the wave record only looks chaotic because there are just so very many waves types coming in simultaneously: each with its own individual waveform, frequency, f-dot plus angles.

Thus in addition to a zillion dollar budget, we also want to make sense of what was recorded. We need a way to pick out individual waves in the presence of very many others. Then we might be able to say for a day of beach watching : out of all that happened that day I found a purely repetitive feature ( template ) of such-and-such frequency, f-dot and there are also two angles which indicate which way I was watching. Send that in a letter.

We do have such a 'pick-out machine'. It's called Einstein At Home, and it was actually invented by a wizard too. Bruce The Wizard to be exact.

Alas the $Z machine has one fussy feature we can't escape. If there is a gap in the record ( say I tripped over the power cord going to the $Z machine and didn't notice for sometime before plugging it back in ) then we can't exactly say what happened during the time when it wasn't recording. You are not allowed to 'make it up' ie. we are doing science. So what we had hoped was going to be a single uninterrupted days worth of data is now two distinct data sets, and crucially we can't say how to match up the wave height at the last instant of the first set with the wave height at the first instant of the second set. Anything could have happened in between.

You can guess the punchlines now. Coherent is when there is no gap in the wave record. Incoherent is when there either is a gap in the record OR the analysis is done on one segment of the record without taking into account any other part(s).

Now Professor Mathematics sez that while a coherent analysis really is the best to pick up those quiet waves, you pay dearly in the difficulty of computation ( however measured ) the longer that coherence time is. Prof Math also says that if one is prepared to take a chance on maybe missing some signals ( making the search less sensitive ), then you can deliberately break up a coherent set into many also coherent but shorter ones. Effectively pretend that you do not know how to join one segment to the next. Analyse each segment separately. Sprinkle in some reasonable probability type thinking to incoherently combine results/findings of the separate segments. A-bing-a-bada-a-boom & a number pops out ( of some lucky E@H contributor machines ) which gauges how likely we are not being fooled when we reckon we have picked out a real wave [ you may need to read that again ].

For E@H one wants to serve the common denominator first, meaning that the commonest machines available are also the weakest computation wise. That's the rationale that limits WU characteristics, and in turn that dictates how you cut up the parameter space ( frequency, f-dot, two angles ) for a given waveform guess ( template ).

Time for a lie down ..... :-))