Task 1714106522

Name: Ter5_1_dns_cfbf00111_segment_6_dms_200_40000_158_1050000_1
Workunit ID: 864552893
Created: 14 Jan 2025 10:18:12 UTC
Sent: 14 Jan 2025 23:24:36 UTC
Report deadline: 28 Jan 2025 23:24:36 UTC
Received: 20 Jan 2025 4:06:35 UTC
Server state: Over
Outcome: Computation error
Client state: Compute error
Exit status: 234 (0x000000EA) Unknown error code
Computer: 12931015
Run time (sec): 12.07
CPU time (sec): 0.61
Peak working set size (MB): 30.41
Peak swap size (MB): 4814.88
Peak disk usage (MB): 153.82
Validation state: Invalid
Granted credit: 0
Application: Binary Radio Pulsar Search (MeerKAT) v0.16 (BRP7-cuda102)
x86_64-pc-linux-gnu
Stderr output

<core_client_version>7.24.1</core_client_version>
<![CDATA[
<message>
process exited with code 234 (0xea, -22)</message>
<stderr_txt>
[08:33:30][17538][INFO ] Application startup - thank you for supporting Einstein@Home!
[08:33:30][17538][INFO ] Starting data processing...
[08:33:30][17538][INFO ] CUDA device ID: 0
[08:33:30][17538][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 7474 MB (394 MB free / 7868 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[08:33:30][17538][INFO ] Using CUDA device #0 "NVIDIA GeForce RTX 3060 Ti" (0 CUDA cores / 0.00 GFLOPS)
[08:33:30][17538][INFO ] Version of installed CUDA driver: 12070
[08:33:30][17538][INFO ] Version of CUDA driver API used: 3020
[08:33:33][17538][INFO ] Number of generated templates to be used: 50000
[08:33:33][17538][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00111_segment_6_dms_200_158.cpt (No such file or directory).
------> Starting from scratch...
[08:33:33][17538][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00111/Ter5_1_dns_cfbf00111_segment_6_dms_200_DM245.80
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776220290689
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174803.15
------> DEC (J2000): -244732.900002
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 245.8 cm^-3 pc
------> Scale factor: 1.00666
[08:33:34][17538][INFO ] Seed for random number generator is -1032170141.
[09:13:25][29506][INFO ] Application startup - thank you for supporting Einstein@Home!
[09:13:25][29506][INFO ] Starting data processing...
[09:13:25][29506][INFO ] CUDA device ID: 0
[09:13:25][29506][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 7382 MB (486 MB free / 7868 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[09:13:25][29506][INFO ] Using CUDA device #0 "NVIDIA GeForce RTX 3060 Ti" (0 CUDA cores / 0.00 GFLOPS)
[09:13:25][29506][INFO ] Version of installed CUDA driver: 12070
[09:13:25][29506][INFO ] Version of CUDA driver API used: 3020
[09:13:28][29506][INFO ] Number of generated templates to be used: 50000
[09:13:28][29506][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00111_segment_6_dms_200_158.cpt (No such file or directory).
------> Starting from scratch...
[09:13:28][29506][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00111/Ter5_1_dns_cfbf00111_segment_6_dms_200_DM245.80
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776220290689
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174803.15
------> DEC (J2000): -244732.900002
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 245.8 cm^-3 pc
------> Scale factor: 1.00666
[09:13:29][29506][INFO ] Seed for random number generator is -1032170141.
[09:13:36][29506][ERROR] Error allocating power spectrum device memory: 50332672 bytes (error: 2)
[09:13:36][29506][ERROR] Demodulation failed (error: 1006)!
[09:13:36][29506][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
[10:54:47][2284][INFO ] Application startup - thank you for supporting Einstein@Home!
[10:54:47][2284][INFO ] Starting data processing...
[10:54:47][2284][INFO ] CUDA device ID: 0
[10:54:47][2284][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 7512 MB (356 MB free / 7868 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[10:54:47][2284][INFO ] Using CUDA device #0 "NVIDIA GeForce RTX 3060 Ti" (0 CUDA cores / 0.00 GFLOPS)
[10:54:47][2284][INFO ] Version of installed CUDA driver: 12070
[10:54:47][2284][INFO ] Version of CUDA driver API used: 3020
[10:54:50][2284][INFO ] Number of generated templates to be used: 50000
[10:54:50][2284][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00111_segment_6_dms_200_158.cpt (No such file or directory).
------> Starting from scratch...
[10:54:50][2284][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00111/Ter5_1_dns_cfbf00111_segment_6_dms_200_DM245.80
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776220290689
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174803.15
------> DEC (J2000): -244732.900002
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 245.8 cm^-3 pc
------> Scale factor: 1.00666
[10:54:51][2284][INFO ] Seed for random number generator is -1032170141.
[10:54:58][2284][ERROR] Error creating CUDA FFT plan (error code: 2)
[10:54:58][2284][ERROR] Demodulation failed (error: 1011)!
[10:54:58][2284][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
[20:11:16][1464][INFO ] Application startup - thank you for supporting Einstein@Home!
[20:11:16][1464][INFO ] Starting data processing...
[20:11:16][1464][INFO ] CUDA device ID: 0
[20:11:16][1464][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 7533 MB (335 MB free / 7868 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:11:16][1464][INFO ] Using CUDA device #0 "NVIDIA GeForce RTX 3060 Ti" (0 CUDA cores / 0.00 GFLOPS)
[20:11:16][1464][INFO ] Version of installed CUDA driver: 12070
[20:11:16][1464][INFO ] Version of CUDA driver API used: 3020
[20:11:19][1464][INFO ] Number of generated templates to be used: 50000
[20:11:19][1464][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00111_segment_6_dms_200_158.cpt (No such file or directory).
------> Starting from scratch...
[20:11:19][1464][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00111/Ter5_1_dns_cfbf00111_segment_6_dms_200_DM245.80
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776220290689
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174803.15
------> DEC (J2000): -244732.900002
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 245.8 cm^-3 pc
------> Scale factor: 1.00666
[20:11:19][1464][INFO ] Seed for random number generator is -1032170141.
[20:11:24][1464][ERROR] Error allocating modulated time series device memory: 100665344 bytes (error: 2)
[20:11:24][1464][ERROR] Demodulation failed (error: 1006)!
[20:11:24][1464][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
[20:14:03][2431][INFO ] Application startup - thank you for supporting Einstein@Home!
[20:14:03][2431][INFO ] Starting data processing...
[20:14:03][2431][INFO ] CUDA device ID: 0
[20:14:03][2431][ERROR] Failed to enable CUDA thread yielding for device #0 (error: 2)! Sorry, will try to occupy one CPU core...
[20:14:13][2431][ERROR] Couldn't acquire CUDA context of device #0 (error: 2)!
[20:14:13][2431][ERROR] Demodulation failed (error: 1002)!
20:14:13 (2431): called boinc_finish(1002)

</stderr_txt>
]]>