Task 1721948046

Stderr output

<core_client_version>7.20.5</core_client_version>
<![CDATA[
<stderr_txt>
putenv 'LAL_DEBUG_LEVEL=3'
2025-01-29 22:26:02.0735 (27281) [normal]: This program is published under the GNU General Public License, version 2
2025-01-29 22:26:02.0736 (27281) [normal]: For details see http://einstein.phys.uwm.edu/license.php
2025-01-29 22:26:02.0736 (27281) [normal]: This Einstein@home App was built at: Mar  7 2024 14:38:26

2025-01-29 22:26:02.0736 (27281) [normal]: Start of BOINC application '../../projects/einstein.phys.uwm.edu/einstein_O3AS_1.16_x86_64-pc-linux-gnu__GW-cuda-3'.
2025-01-29 22:26:02.0743 (27281) [debug]: Flags: LAL_DEBUG, OPTIMIZE, HS_OPTIMIZATION, GC_SSE2_OPT, CUDA, X64, SSE, SSE2, GNUC X86 GNUX86
2025-01-29 22:26:02.0743 (27281) [debug]: glibc version/release: 2.36/stable
2025-01-29 22:26:02.74397 - mytime()
2025-01-29 22:26:02.0745 (27281) [debug]: Set up communication with graphics process.
2025-01-29 22:26:02.0860 (27281) [normal]: Parsed user input successfully
Code-version: %% LAL: 7.1.4.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALPulsar: 3.1.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALApps: 7.3.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)

2025-01-29 22:26:02.0860 (27281) [normal]: Initialise compartments with freqWidth = 0.05 and candidates per compartment = 10000.
2025-01-29 22:26:02.0862 (27281) [debug]: Opening temp output file '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0__O3ASBu_179.00Hz_24763_0_0.tmp' for writing ...
2025-01-29 22:26:02.3225 (27281) [normal]: Reading input data ... 
2025-01-29 22:26:02.3225 (27281) [normal]: Loading SFTs matching '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0180.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0180.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0180.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0180.20_O3aLC01Cl1In0' into catalog ...2025-01-29 22:26:02.5321 (27281) [normal]: done.
2025-01-29 22:26:02.5321 (27281) [normal]: Validating SFTs (detectors: H1, L1, ) ... success.
2025-01-29 22:26:04.8681 (27281) [normal]: Search FstatMethod used: 'ResampGPU'
2025-01-29 22:26:04.8681 (27281) [normal]: Recalc FstatMethod used: 'DemodGPU'
2025-01-29 22:26:04.8681 (27281) [normal]: GPU Device used for Search/Recalc and/or semi coherent step: 'NVIDIA GeForce GTX 980'
2025-01-29 22:26:04.8681 (27281) [normal]: GPU Backend used for Search/Recalc and/or semi coherent step: 'CUDA'
2025-01-29 22:26:04.8681 (27281) [normal]: GPU version is used for the semi-coherent step!
2025-01-29 22:26:18.4136 (27281) [normal]: Number of segments: 18, total number of SFTs in segments: 5748
2025-01-29 22:26:18.4175 (27281) [normal]: Finished reading input data.
2025-01-29 22:26:18.4176 (27281) [normal]: GPS reference time = 1246070525.0000 ,  GPS data mid time = 1246070525.0000
2025-01-29 22:26:18.4176 (27281) [normal]: dFreqStack = 1.000000e-06, df1dot = 7.500000e-11, df2dot = 0.000000e+00, df3dot = 0.000000e+00
2025-01-29 22:26:18.4176 (27281) [normal]: Setup, N = 18, T = 878400 s, Tobs = 15809012 s, gammaRefine = 250, gamma2Refine = 1098, gamma3Refine = 1

DEPRECATION WARNING: program has invoked obsolete function InitDopplerSkyScan(). Please see XLALInitDopplerSkyScan() for information about a replacement.
2025-01-29 22:26:30.2375 (27281) [normal]: INFO: No checkpoint checkpoint.cpt found - starting from scratch
2025-01-29 22:26:30.2823 (27281) [normal]: Cpt:0,  total:9000,  sky:1/225,  f1dot:1/40

0.2025-01-29 22:26:30.2829 (27281) [normal]: CG:18010710 FG:1000000  f1dotmin_fg:-2.73735059761e-09 df1dot_fg:2.988047808765e-13 f2dotmin_fg:0 df2dot_fg:0 f3dotmin_fg:0 df3dot_fg:1
.......................................
1................c
........................
2........................................
3........................................
4.....c
...................................
5........................................
6...................................c
.....
7........................................
8........................................
9..........................c
..............
10........................................
11........................................
12..................c
......................
13........................................
14........................................
15..........c
..............................
16........................................
17........................................
18..c
......................................
19........................................
20..................................c
......
21........................................
22........................................
23..........................c
..............
24........................................
25........................................
26...................c
.....................
27........................................
28........................................
29...........c
.............................
30........................................
31........................................
32....c
....................................
33........................................
34.....................................c
...
35............................
2025-01-29 22:39:21.858318
-- signal handler called: signal 15
XLAL Error - XLALGPUMarkStreamsAsFree (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/GPUUtils.c:1961): GPU is not initialized!
XLAL Error - XLALGPUMarkStreamsAsFree (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/GPUUtils.c:1961): Generic failure
XLAL Error - XLALGPUWait (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/GPUUtils.c:476): Check failed: XLALGPUMarkStreamsAsFree(((void *)0)) == XLAL_SUCCESS
XLAL Error - XLALGPUWait (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/GPUUtils.c:476): Internal function call failed: Generic failure
XLAL Error - XLALSemiCohStep_GPU (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:4677): SemiCohStep
XLAL Error - XLALSemiCohStep_GPU (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:4677): Internal function call failed: Generic failure
XLAL Error - MAIN (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:2308): Check failed: XLALSemiCohStep_GPU( &coarsegrid, &finegrid, nStacks, &usefulParams, NSegmentsInv, uvar->SortToplist, compartment, compartment2, compartment3) == XLAL_SUCCESS
XLAL Error - MAIN (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:2308): Internal function call failed: Generic failure
2025-01-29 22:39:21.8593 (27281) [CRITICAL]: ERROR: MAIN() returned with error '-1'
Code-version: %% LAL: 7.1.4.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALPulsar: 3.1.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALApps: 7.3.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)

FPU status flags: 
2025-01-29 22:39:21.8598 (27281) [debug]: worker done. return(-1) to caller
2025-01-29 22:39:21.8598 (27281) [normal]: done. calling boinc_finish(-1).
22:39:21 (27281): called boinc_finish
XLALCUDARelease: Total malloc from <TYPE>VectorGPU is greater than 0: 2076648813XLALCUDARelease: Total malloc from COMPLEX8FFTPlanGPU is greater than 0: 671088649 stack frames obtained for this thread:
Use gdb command: 'info line *0xADDRESS' to print corresponding line numbers.
../../projects/einstein.phys.uwm.edu/einstein_O3AS_1.16_x86_64-pc-linux-gnu__GW-cuda-3[0x4301f7]
/lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7fc642f4b050]
/lib/x86_64-linux-gnu/libc.so.6(clock_nanosleep+0x65)[0x7fc642fde505]
/lib/x86_64-linux-gnu/libc.so.6(nanosleep+0x13)[0x7fc642fe2e13]
/lib/x86_64-linux-gnu/libc.so.6(usleep+0x45)[0x7fc64300e205]
../../projects/einstein.phys.uwm.edu/einstein_O3AS_1.16_x86_64-pc-linux-gnu__GW-cuda-3[0x66942d]
../../projects/einstein.phys.uwm.edu/einstein_O3AS_1.16_x86_64-pc-linux-gnu__GW-cuda-3[0x657a2d]
/lib/x86_64-linux-gnu/libc.so.6(+0x891c4)[0x7fc642f981c4]
/lib/x86_64-linux-gnu/libc.so.6(+0x10985c)[0x7fc64301885c]

End of stacktrace
Stack trace of LAL functions in worker thread:
InitDopplerSkyScan at /home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/DopplerScan.c:291
At lowest level status code = 0: NO LAL ERROR REGISTERED
putenv 'LAL_DEBUG_LEVEL=3'
2025-01-29 23:34:06.5488 (1210) [normal]: This program is published under the GNU General Public License, version 2
2025-01-29 23:34:06.5488 (1210) [normal]: For details see http://einstein.phys.uwm.edu/license.php
2025-01-29 23:34:06.5488 (1210) [normal]: This Einstein@home App was built at: Mar  7 2024 14:38:26

2025-01-29 23:34:06.5488 (1210) [normal]: Start of BOINC application '../../projects/einstein.phys.uwm.edu/einstein_O3AS_1.16_x86_64-pc-linux-gnu__GW-cuda-3'.
2025-01-29 23:34:06.5522 (1210) [debug]: Flags: LAL_DEBUG, OPTIMIZE, HS_OPTIMIZATION, GC_SSE2_OPT, CUDA, X64, SSE, SSE2, GNUC X86 GNUX86
2025-01-29 23:34:06.5523 (1210) [debug]: glibc version/release: 2.36/stable
2025-01-29 23:34:06.552312 - mytime()
2025-01-29 23:34:06.5525 (1210) [debug]: Set up communication with graphics process.
2025-01-29 23:34:06.5635 (1210) [normal]: Parsed user input successfully
Code-version: %% LAL: 7.1.4.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALPulsar: 3.1.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALApps: 7.3.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)

2025-01-29 23:34:06.5636 (1210) [normal]: Initialise compartments with freqWidth = 0.05 and candidates per compartment = 10000.
2025-01-29 23:34:06.5638 (1210) [debug]: Opening temp output file '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0__O3ASBu_179.00Hz_24763_0_0.tmp' for writing ...
2025-01-29 23:34:06.7643 (1210) [normal]: Reading input data ... 
2025-01-29 23:34:06.7643 (1210) [normal]: Loading SFTs matching '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0180.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0180.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0180.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0180.20_O3aLC01Cl1In0' into catalog ...2025-01-29 23:34:06.9205 (1210) [normal]: done.
2025-01-29 23:34:06.9205 (1210) [normal]: Validating SFTs (detectors: H1, L1, ) ... success.
2025-01-29 23:34:08.8763 (1210) [normal]: Search FstatMethod used: 'ResampGPU'
2025-01-29 23:34:08.8763 (1210) [normal]: Recalc FstatMethod used: 'DemodGPU'
2025-01-29 23:34:08.8763 (1210) [normal]: GPU Device used for Search/Recalc and/or semi coherent step: 'NVIDIA GeForce GTX 980'
2025-01-29 23:34:08.8763 (1210) [normal]: GPU Backend used for Search/Recalc and/or semi coherent step: 'CUDA'
2025-01-29 23:34:08.8763 (1210) [normal]: GPU version is used for the semi-coherent step!
2025-01-29 23:34:22.4035 (1210) [normal]: Number of segments: 18, total number of SFTs in segments: 5748
2025-01-29 23:34:22.4074 (1210) [normal]: Finished reading input data.
2025-01-29 23:34:22.4074 (1210) [normal]: GPS reference time = 1246070525.0000 ,  GPS data mid time = 1246070525.0000
2025-01-29 23:34:22.4074 (1210) [normal]: dFreqStack = 1.000000e-06, df1dot = 7.500000e-11, df2dot = 0.000000e+00, df3dot = 0.000000e+00
2025-01-29 23:34:22.4074 (1210) [normal]: Setup, N = 18, T = 878400 s, Tobs = 15809012 s, gammaRefine = 250, gamma2Refine = 1098, gamma3Refine = 1

DEPRECATION WARNING: program has invoked obsolete function InitDopplerSkyScan(). Please see XLALInitDopplerSkyScan() for information about a replacement.
2025-01-29 23:34:33.4316 (1210) [normal]: Successfully read checkpoint:1397
2025-01-29 23:34:33.5012 (1210) [normal]: Cpt:1397,  total:9000,  sky:35/225,  f1dot:38/40

34.2025-01-29 23:34:33.5017 (1210) [normal]: CG:18010710 FG:1000000  f1dotmin_fg:3.764940239044e-11 df1dot_fg:2.988047808765e-13 f2dotmin_fg:0 df2dot_fg:0 f3dotmin_fg:0 df3dot_fg:1
..
35........................................
36....................c
....................
37........................................
38........................................
39..............c
..........................
40........................................
41........................................
42........c
................................
43........................................
44........................................
45...c
.....................................
46........................................
47.....................................c
...
48........................................
49........................................
50................................c
........
51........................................
52........................................
53...........................c
.............
54........................................
55........................................
56......................c
..................
57........................................
58........................................
59.................c
.......................
60........................................
61........................................
62.............c
...........................
63........................................
64........................................
65.......c
.................................
66........................................
67........................................
68..c
......................................
69........................................
70......................................c
..
71........................................
72........................................
73.................................c
.......
74........................................
75........................................
76............................c
............
77........................................
78........................................
79.......................c
.................
80........................................
81........................................
82..................c
......................
83........................................
84........................................
85..............c
..........................
86........................................
87........................................
88...........c
.............................
89........................................
90........................................
91.......c
.................................
92........................................
93........................................
94..c
......................................
95........................................
96......................................c
..
97........................................
98........................................
99.................................c
.......
100........................................
101........................................
102............................c
............
103........................................
104........................................
105.......................c
.................
106........................................
107........................................
108...................c
.....................
109........................................
110........................................
111..............c
..........................
112........................................
113........................................
114.........c
...............................
115........................................
116........................................
117......c
..................................
118........................................
119........................................
120..c
......................................
121........................................
122......................................c
..
123........................................
124........................................
125..................................c
......
126........................................
127........................................
128................................c
........
129........................................
130........................................
131.............................c
...........
132........................................
133........................................
134.........................c
...............
135........................................
136........................................
137.....................c
...................
138........................................
139........................................
140.................c
.......................
141........................................
142........................................
143.............c
...........................
144........................................
145........................................
146..........c
..............................
147........................................
148........................................
149......c
..................................
150........................................
151........................................
152..c
......................................
153........................................
154........................................c

155........................................
156........................................
157......................................c
..
158........................................
159........................................
160...................................c
.....
161........................................
162........................................
163................................c
........
164........................................
165........................................
166.............................c
...........
167........................................
168........................................
169.........................c
...............
170........................................
171........................................
172.....................c
...................
173........................................
174........................................
175.................c
.......................
176........................................
177........................................
178.............c
...........................
179........................................
180........................................
181.........c
...............................
182........................................
183........................................
184.....c
...................................
185........................................
186........................................
187.c
.......................................
188........................................
189......................................c
..
190........................................
191........................................
192...................................c
.....
193........................................
194........................................
195...............................c
.........
196........................................
197........................................
198...........................c
.............
199........................................
200........................................
201.......................c
.................
202........................................
203........................................
204...................c
.....................
205........................................
206........................................
207...............c
.........................
208........................................
209........................................
210...........c
.............................
211........................................
212........................................
213.......c
.................................
214........................................
215........................................
216...c
.....................................
217........................................
218........................................c

219........................................
220........................................
221......................................c
..
222........................................
223........................................
224.................................c
.......
2025-01-30 00:40:28.0012 (1210) [normal]: Finished main analysis.
2025-01-30 00:40:28.0012 (1210) [normal]: Recalculating statistics for the final toplist(s)...
2025-01-30 00:40:31.7642 (1210) [normal]: Recalculation step: 20000/200000
2025-01-30 00:40:35.3582 (1210) [normal]: Recalculation step: 40000/200000
2025-01-30 00:40:38.9265 (1210) [normal]: Recalculation step: 60000/200000
2025-01-30 00:40:42.5190 (1210) [normal]: Recalculation step: 80000/200000
2025-01-30 00:40:46.1045 (1210) [normal]: Recalculation step: 100000/200000
2025-01-30 00:40:49.6915 (1210) [normal]: Recalculation step: 120000/200000
2025-01-30 00:40:53.2765 (1210) [normal]: Recalculation step: 140000/200000
2025-01-30 00:40:56.8672 (1210) [normal]: Recalculation step: 160000/200000
2025-01-30 00:41:00.4650 (1210) [normal]: Recalculation step: 180000/200000
2025-01-30 00:41:04.0631 (1210) [normal]: Recalculation step: 200000/200000
2025-01-30 00:41:04.2583 (1210) [normal]: Finished recalculating toplist statistics.
2025-01-30 00:41:04.2583 (1210) [normal]: Finished in 4017.69 s with peak RAM usage: 1337.8 MB on CPU 'AMD Ryzen 7 2700X Eight-Core Processor         ', peak VRAM usage: 2044.4 MB on GPU Device: 'NVIDIA GeForce GTX 980' with backend: 'CUDA'.
2025-01-30 00:41:04.2895 (1210) [debug]: Writing output ... Closing temp output file '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0__O3ASBu_179.00Hz_24763_0_0.tmp' ... renaming temp output file '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0__O3ASBu_179.00Hz_24763_0_0.tmp' to '../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0__O3ASBu_179.00Hz_24763_0_0' ... done.
2025-01-30 00:41:04.5110 (1210) [normal]: Restarted from checkpoint 1397

DEPRECATION WARNING: program has invoked obsolete function FreeDopplerSkyScan(). Please see XLALDestroyDopplerSkyScan() for information about a replacement.
Code-version: %% LAL: 7.1.4.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALPulsar: 3.1.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)
%% LALApps: 7.3.0.1 (UNCLEAN d065dd0b1daca1f30b01d44a0f807e06f37b4958)

FPU status flags: 
2025-01-30 00:41:04.6079 (1210) [debug]: worker done. return(0) to caller
2025-01-30 00:41:04.6079 (1210) [normal]: done. calling boinc_finish(0).
00:41:04 (1210): called boinc_finish

</stderr_txt>
]]>