Task 1720114281

Name: h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1
Workunit ID: 868224317
Created: 25 Jan 2025 22:52:46 UTC
Sent: 25 Jan 2025 22:52:51 UTC
Report deadline: 1 Feb 2025 22:52:51 UTC
Received: 26 Jan 2025 6:09:59 UTC
Server state: Over
Outcome: Computation error
Client state: Compute error
Exit status: 255 (0x000000FF) Unknown error code
Computer: 13201321
Run time (sec): 1,264.98
CPU time (sec): 1,262.36
Peak working set size (MB): 914.73
Peak swap size (MB): 7227.26
Peak disk usage (MB): 18.89
Validation state: Invalid
Granted credit: 0
Application: All-Sky Gravitational Wave search on O3
Anonymous platform
Stderr output

<core_client_version>7.19.0</core_client_version>
<![CDATA[
<message>
process exited with code 255 (0xff, -1)</message>
<stderr_txt>
putenv 'LAL_DEBUG_LEVEL=3'
2025-01-26 00:47:27.7383 (291601) [normal]: This program is published under the GNU General Public License, version 2
2025-01-26 00:47:27.7384 (291601) [normal]: For details see http://einstein.phys.uwm.edu/license.php
2025-01-26 00:47:27.7384 (291601) [normal]: This Einstein@home App was built at: Feb 23 2024 05:11:10

2025-01-26 00:47:27.7384 (291601) [normal]: Start of BOINC application '../../projects/einstein.phys.uwm.edu/GW_einstein_O3AS_1.17_x86_64-pc-linux-gnu__GW-cuda'.
2025-01-26 00:47:27.7386 (291601) [debug]: Flags: LAL_DEBUG, OPTIMIZE, HS_OPTIMIZATION, GC_SSE2_OPT, X64, SSE, SSE2, GNUC X86 GNUX86
2025-01-26 00:47:27.7386 (291601) [debug]: glibc version/release: 2.35/stable
2025-01-26 00:47:27.738624 - mytime()
2025-01-26 00:47:27.7386 (291601) [debug]: Set up communication with graphics process.
2025-01-26 00:47:27.7424 (291601) [normal]: Parsed user input successfully
Code-version: %% LAL: 7.1.4.1 (CLEAN )
%% LALPulsar: 3.1.0.1 (CLEAN )
%% LALApps: 7.3.0.1 (UNCLEAN )

2025-01-26 00:47:27.7424 (291601) [normal]: Initialise compartments with freqWidth = 0.05 and candidates per compartment = 10000.
2025-01-26 00:47:27.7424 (291601) [debug]: Opening temp output file '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1_0.tmp' for writing ...
2025-01-26 00:47:27.8122 (291601) [normal]: Reading input data ... 
2025-01-26 00:47:27.8122 (291601) [normal]: Loading SFTs matching '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0177.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.20_O3aLC01Cl1In0' into catalog ...2025-01-26 00:47:27.8826 (291601) [normal]: done.
2025-01-26 00:47:27.8826 (291601) [normal]: Validating SFTs (detectors: H1, L1, ) ... success.
2025-01-26 00:47:28.7034 (291601) [normal]: Search FstatMethod used: 'ResampGPU'
2025-01-26 00:47:28.7034 (291601) [normal]: Recalc FstatMethod used: 'DemodSSE'
2025-01-26 00:47:28.7034 (291601) [normal]: GPU Device used for Search/Recalc and/or semi coherent step: 'NVIDIA TITAN V'
2025-01-26 00:47:28.7034 (291601) [normal]: GPU Backend used for Search/Recalc and/or semi coherent step: 'CUDA'
2025-01-26 00:47:28.7034 (291601) [normal]: GPU version is used for the semi-coherent step!
2025-01-26 00:47:31.7572 (291601) [normal]: Number of segments: 18, total number of SFTs in segments: 5748
2025-01-26 00:47:31.7584 (291601) [normal]: Finished reading input data.
2025-01-26 00:47:31.7584 (291601) [normal]: GPS reference time = 1246070525.0000 ,  GPS data mid time = 1246070525.0000
2025-01-26 00:47:31.7584 (291601) [normal]: dFreqStack = 1.000000e-06, df1dot = 7.500000e-11, df2dot = 0.000000e+00, df3dot = 0.000000e+00
2025-01-26 00:47:31.7584 (291601) [normal]: Setup, N = 18, T = 878400 s, Tobs = 15809012 s, gammaRefine = 250, gamma2Refine = 1098, gamma3Refine = 1

DEPRECATION WARNING: program has invoked obsolete function InitDopplerSkyScan(). Please see XLALInitDopplerSkyScan() for information about a replacement.
2025-01-26 00:47:35.4853 (291601) [normal]: INFO: No checkpoint checkpoint.cpt found - starting from scratch
2025-01-26 00:47:35.4933 (291601) [normal]: Cpt:0,  total:9000,  sky:1/225,  f1dot:1/40

0.2025-01-26 00:47:35.4935 (291601) [normal]: CG:9010710 FG:500000  f1dotmin_fg:-2.73735059761e-09 df1dot_fg:2.988047808765e-13 f2dotmin_fg:0 df2dot_fg:0 f3dotmin_fg:0 df3dot_fg:1
.......................................
1........................................
2........................................
3........................................
4........................................
5........................................
6........................................
7........................................
8........................................
9........................................
10........................................
11........................................
12........................................
13.........................c
...............
14........................................
15........................................
16........................................
17........................................
18........................................
19........................................
20........................................
21........................................
22........................................
23........................................
24........................................
25........................................
26........................................
27........................................
28........................................
29........................................
30.........................c
...............
31........................................
32........................................
33........................................
34........................................
35........................................
36........................................
37........................................
38........................................
39........................................
40........................................
41........................................
42........................................
43........................................
44........................................
45........................................
46........................................
47........................................
48.....c
...................................
49........................................
50........................................
51........................................
52........................................
53........................................
54........................................
55........................................
56........................................
57........................................
58........................................
59........................................
60........................................
61........................................
62........................................
63........................................
64........................................
65.................................c
.......
66........................................
67........................................
68........................................
69........................................
70........................................
71........................................
72........................................
73........................................
74........................................
75........................................
76........................................
77........................................
78........................................
79........................................
80........................................
81........................................
82........................................
83..........................c
..............
84........................................
85........................................
86........................................
87........................................
88........................................
89........................................
90........................................
91........................................
92........................................
93........................................
94........................................
95........................................
96........................................
97........................................
98........................................
99........................................
100........................................
101..........................c
..............
102........................................
103........................................
104........................................
105........................................
106........................................
107........................................
108........................................
109........................................
110........................................
111........................................
112........................................
113........................................
114........................................
115........................................
116........................................
117........................................
118........................................
119.........................c
...............
120........................................
121........................................
122........................................
123........................................
124........................................
125........................................
126........................................
127........................................
128........................................
129........................................
130........................................
131........................................
132........................................
133........................................
134........................................
135........................................
136........................................
137..........................c
..............
138........................................
139........................................
140........................................
141........................................
142........................................
143........................................
144........................................
145........................................
146........................................
147........................................
148........................................
149........................................
150........................................
151........................................
152........................................
153........................................
154........................................
155...............................c
.........
156........................................
157........................................
158........................................
159........................................
160........................................
161........................................
162........................................
163........................................
164........................................
165........................................
166........................................
167........................................
168........................................
169........................................
170........................................
171........................................
172........................................
173.................................c
.......
174........................................
175........................................
176........................................
177........................................
178........................................
179........................................
180........................................
181........................................
182........................................
183........................................
184........................................
185........................................
186........................................
187........................................
188........................................
189........................................
190........................................
191...................................c
.....
192........................................
193........................................
194........................................
195........................................
196........................................
197........................................
198........................................
199........................................
200........................................
201........................................
202........................................
203........................................
204........................................
205........................................
206........................................
207........................................
208........................................
209........................................c

210........................................
211........................................
212........................................
213........................................
214........................................
215........................................
216........................................
217........................................
218........................................
219........................................
220........................................
221........................................
222........................................
223........................................
224........................................
2025-01-26 01:00:18.9678 (291601) [normal]: Finished main analysis.
2025-01-26 01:00:18.9679 (291601) [normal]: Recalculating statistics for the final toplist(s)...
2025-01-26 01:00:34.2002 (291601) [normal]: Finished recalculating toplist statistics.
2025-01-26 01:00:34.2003 (291601) [normal]: Finished in 786.46 s with peak RAM usage: 1303.6 MB on CPU '<unknown>', peak VRAM usage: 931.8 MB on GPU Device: 'NVIDIA TITAN V' with backend: 'CUDA'.
2025-01-26 01:00:34.2138 (291601) [debug]: Writing output ... Closing temp output file '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1_0.tmp' ... renaming temp output file '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1_0.tmp' to '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1_0' ... done.

DEPRECATION WARNING: program has invoked obsolete function FreeDopplerSkyScan(). Please see XLALDestroyDopplerSkyScan() for information about a replacement.
2025-01-26 01:00:34.2748 (291601) [normal]: Parsed user input successfully
Code-version: %% LAL: 7.1.4.1 (CLEAN )
%% LALPulsar: 3.1.0.1 (CLEAN )
%% LALApps: 7.3.0.1 (UNCLEAN )

2025-01-26 01:00:34.2749 (291601) [normal]: Initialise compartments with freqWidth = 0.05 and candidates per compartment = 10000.
2025-01-26 01:00:34.2749 (291601) [debug]: Opening temp output file '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0__O3ASBuB_178.00Hz_6754_1_1.tmp' for writing ...
2025-01-26 01:00:34.3374 (291601) [normal]: Reading input data ... 
2025-01-26 01:00:34.3374 (291601) [normal]: Loading SFTs matching '../../projects/einstein.phys.uwm.edu/h1_0177.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0177.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.40_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.60_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0178.80_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.00_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/h1_0179.20_O3aLC01Cl1In0;../../projects/einstein.phys.uwm.edu/l1_0179.20_O3aLC01Cl1In0' into catalog ...2025-01-26 01:00:34.4103 (291601) [normal]: done.
2025-01-26 01:00:34.4103 (291601) [normal]: Validating SFTs (detectors: H1, L1, ) ... success.
2025-01-26 01:00:35.1758 (291601) [normal]: Search FstatMethod used: 'ResampGPU'
2025-01-26 01:00:35.1758 (291601) [normal]: Recalc FstatMethod used: 'DemodSSE'
2025-01-26 01:00:35.1758 (291601) [normal]: GPU Device used for Search/Recalc and/or semi coherent step: 'NVIDIA TITAN V'
2025-01-26 01:00:35.1758 (291601) [normal]: GPU Backend used for Search/Recalc and/or semi coherent step: 'CUDA'
2025-01-26 01:00:35.1758 (291601) [normal]: GPU version is used for the semi-coherent step!
2025-01-26 01:00:37.6440 (291601) [normal]: Number of segments: 18, total number of SFTs in segments: 5748
2025-01-26 01:00:37.6453 (291601) [normal]: Finished reading input data.
2025-01-26 01:00:37.6453 (291601) [normal]: GPS reference time = 1246070525.0000 ,  GPS data mid time = 1246070525.0000
2025-01-26 01:00:37.6454 (291601) [normal]: dFreqStack = 1.000000e-06, df1dot = 7.500000e-11, df2dot = 0.000000e+00, df3dot = 0.000000e+00
2025-01-26 01:00:37.6454 (291601) [normal]: Setup, N = 18, T = 878400 s, Tobs = 15809012 s, gammaRefine = 250, gamma2Refine = 1098, gamma3Refine = 1
2025-01-26 01:00:41.3695 (291601) [normal]: INFO: No checkpoint checkpoint.cpt found - starting from scratch
2025-01-26 01:00:41.3770 (291601) [normal]: Cpt:0,  total:9000,  sky:1/225,  f1dot:1/40

0.2025-01-26 01:00:41.3772 (291601) [normal]: CG:9010710 FG:500000  f1dotmin_fg:-2.73735059761e-09 df1dot_fg:2.988047808765e-13 f2dotmin_fg:0 df2dot_fg:0 f3dotmin_fg:0 df3dot_fg:1
c
.......................................
1........................................
2........................................
3........................................
4........................................
5........................................
6........................................
7........................................
8........................................
9........................................
10........................................
11........................................
12........................................
13........................................
14........................................
15.................................c
.......
16........................................
17........................................
18........................................
19........................................
20........................................
21........................................
22........................................
23........................................
24........................................
25........................................
26........................................
27........................................
28........................................
29........................................
30........................................
31........................................
32....................................c
....
33........................................
34........................................
35........................................
36........................................
37........................................
38........................................
39........................................
40........................................
41........................................
42........................................
43........................................
44........................................
45........................................
46........................................
47........................................
48........................................
49........................................
50...............c
.........................
51........................................
52........................................
53........................................
54........................................
55........................................
56........................................
57........................................
58........................................
59........................................
60........................................
61........................................
62........................................
63........................................
64........................................
65........................................
66........................................
67....................................c
....
68........................................
69........................................
70........................................
71........................................
72........................................
73........................................
74........................................
75........................................
76........................................
77........................................
78........................................
79........................................
80........................................
81........................................
82........................................
83........................................
84........................................
85...............................c
.........
86........................................
87........................................
88........................................
89........................................
90........................................
91........................................
92........................................
93........................................
94........................................
95........................................
96........................................
97........................................
98........................................
99........................................
100........................................
101........................................
102........................................
103............................c
............
104........................................
105........................................
106........................................
107........................................
108........................................
109........................................
110........................................
111........................................
112........................................
113........................................
114........................................
115........................................
116........................................
117........................................
118........................................
119........................................
120........................................
121...........................c
.............
122........................................
123........................................
124........................................
125........................................
126........................................
127........................................
128........................................
129........................................
130........................................
131........................................
132........................................
133........................................
134........................................
135........................................
136..........XLAL Error - XLALCUDAMemcpy (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/CUDAUtils.c:144): Moving CUDA memory in 1 direction failed with CUDA error: cudaErrorInvalidValue
XLAL Error - XLALCUDAMemcpy (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/CUDAUtils.c:144): Generic failure
XLAL Error - XLALCUDACheckErrorsBeforeKernelLaunch (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/CUDAUtils.c:177): cudaGetLastError() failed with CUDA error: cudaErrorInvalidValue
XLAL Warning - XLALSetErrno (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lal/lib/std/XLALError.c:349): Ignoring previous error (xlalErrno=128) Generic failure

XLAL Error - XLALCUDACheckErrorsBeforeKernelLaunch (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalpulsar/lib/GPUUtils/CUDAUtils.c:177): Generic failure
XLAL Error - XLALSemiCohStep_CUDA (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT_CUDA.cu:478): Check failed: XLALCUDACheckErrorsBeforeKernelLaunch() == XLAL_SUCCESS
XLAL Error - XLALSemiCohStep_CUDA (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT_CUDA.cu:478): Internal function call failed: Generic failure
XLAL Error - XLALSemiCohStep_GPU (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:4632): Check failed: (*usefulparams->gct_gpu_funcs->SemiCohStep) ( coarsegrid, finegrid, stacks, NSegmentsInv, toplists_sortby, usefulparams->BSGLsetupGPU, usefulparams->computeBSGL, usefulparams->getMaxFperSeg, toplist1_last_entryGPU->data, toplist2_last_entryGPU->data, toplist3_last_entryGPU->data ) == XLAL_SUCCESS
XLAL Error - XLALSemiCohStep_GPU (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:4632): Internal function call failed: Generic failure
XLAL Error - MAIN (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:2284): Check failed: XLALSemiCohStep_GPU( &coarsegrid, &finegrid, nStacks, &usefulParams, NSegmentsInv, uvar->SortToplist, compartment, compartment2, compartment3) == XLAL_SUCCESS
XLAL Error - MAIN (/home/boinc/lalsuite/EinsteinAtHome/source/lalsuite/lalapps/src/pulsar/GCT/HierarchSearchGCT.c:2284): Internal function call failed: Generic failure
2025-01-26 01:08:30.3999 (291601) [CRITICAL]: ERROR: MAIN() returned with error '-1'
Code-version: %% LAL: 7.1.4.1 (CLEAN )
%% LALPulsar: 3.1.0.1 (CLEAN )
%% LALApps: 7.3.0.1 (UNCLEAN )

FPU status flags:  COND_1 PRECISION
2025-01-26 01:08:30.4001 (291601) [debug]: worker done. return(-1) to caller
2025-01-26 01:08:30.4001 (291601) [normal]: done. calling boinc_finish(-1).
01:08:30 (291601): called boinc_finish

</stderr_txt>
]]>