Task 1720802504

Stderr output

<core_client_version>8.0.2</core_client_version>
<![CDATA[
<stderr_txt>

------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[15:11:08][6312][INFO ] Seed for random number generator is -1030801420.
[15:11:13][6312][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[15:11:13][6312][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[15:11:13][6312][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[15:11:13][6312][ERROR] Demodulation failed (error: 1006)!
[15:11:13][6312][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[15:26:19][15192][INFO ] Starting data processing...
[15:26:20][15192][INFO ] CUDA device ID: 0
[15:26:20][15192][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:26:20][15192][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:26:20][15192][INFO ] Version of installed CUDA driver: 9010
[15:26:20][15192][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[15:33:08][11724][INFO ] Starting data processing...
[15:33:08][11724][INFO ] CUDA device ID: 0
[15:33:08][11724][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:33:08][11724][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:33:08][11724][INFO ] Version of installed CUDA driver: 9010
[15:33:08][11724][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[15:33:58][6400][INFO ] Starting data processing...
[15:33:58][6400][INFO ] CUDA device ID: 0
[15:33:58][6400][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:33:58][6400][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:33:58][6400][INFO ] Version of installed CUDA driver: 9010
[15:33:58][6400][INFO ] Version of CUDA driver API used: 3020
[15:34:32][6400][INFO ] Number of generated templates to be used: 50000
[15:34:32][6400][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[15:34:32][6400][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[15:34:32][6400][INFO ] Seed for random number generator is -1030801420.
[15:34:37][6400][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[15:34:37][6400][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[15:34:37][6400][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[15:34:37][6400][ERROR] Demodulation failed (error: 1006)!
[15:34:37][6400][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[15:49:47][3964][INFO ] Starting data processing...
[15:49:48][3964][INFO ] CUDA device ID: 0
[15:49:48][3964][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:49:48][3964][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:49:48][3964][INFO ] Version of installed CUDA driver: 9010
[15:49:48][3964][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[15:50:08][13856][INFO ] Starting data processing...
[15:50:08][13856][INFO ] CUDA device ID: 0
[15:50:08][13856][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:50:08][13856][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:50:08][13856][INFO ] Version of installed CUDA driver: 9010
[15:50:08][13856][INFO ] Version of CUDA driver API used: 3020
[15:50:42][13856][INFO ] Number of generated templates to be used: 50000
[15:50:42][13856][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[15:50:42][13856][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[15:50:42][13856][INFO ] Seed for random number generator is -1030801420.
[15:50:47][13856][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[15:50:47][13856][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[15:50:47][13856][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[15:50:47][13856][ERROR] Demodulation failed (error: 1006)!
[15:50:47][13856][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[15:51:07][10180][INFO ] Starting data processing...
[15:51:08][10180][INFO ] CUDA device ID: 0
[15:51:08][10180][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[15:51:08][10180][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[15:51:08][10180][INFO ] Version of installed CUDA driver: 9010
[15:51:08][10180][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[16:06:29][12184][INFO ] Starting data processing...
[16:06:29][12184][INFO ] CUDA device ID: 0
[16:06:29][12184][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[16:06:29][12184][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[16:06:29][12184][INFO ] Version of installed CUDA driver: 9010
[16:06:29][12184][INFO ] Version of CUDA driver API used: 3020
[16:07:03][12184][INFO ] Number of generated templates to be used: 50000
[16:07:03][12184][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[16:07:03][12184][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[16:07:03][12184][INFO ] Seed for random number generator is -1030801420.
[16:07:08][12184][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[16:07:08][12184][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[16:07:08][12184][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[16:07:08][12184][ERROR] Demodulation failed (error: 1006)!
[16:07:08][12184][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[16:22:18][10144][INFO ] Starting data processing...
[16:22:18][10144][INFO ] CUDA device ID: 0
[16:22:18][10144][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[16:22:18][10144][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[16:22:18][10144][INFO ] Version of installed CUDA driver: 9010
[16:22:18][10144][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[16:24:17][9884][INFO ] Starting data processing...
[16:24:17][9884][INFO ] CUDA device ID: 0
[16:24:17][9884][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[16:24:17][9884][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[16:24:17][9884][INFO ] Version of installed CUDA driver: 9010
[16:24:17][9884][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[16:25:00][1460][INFO ] Starting data processing...
[16:25:01][1460][INFO ] CUDA device ID: 0
[16:25:01][1460][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[16:25:01][1460][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[16:25:01][1460][INFO ] Version of installed CUDA driver: 9010
[16:25:01][1460][INFO ] Version of CUDA driver API used: 3020
[16:25:35][1460][INFO ] Number of generated templates to be used: 50000
[16:25:35][1460][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[16:25:35][1460][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[16:25:35][1460][INFO ] Seed for random number generator is -1030801420.
[16:25:40][1460][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[16:25:40][1460][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[16:25:40][1460][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[16:25:40][1460][ERROR] Demodulation failed (error: 1006)!
[16:25:40][1460][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[18:10:17][4976][INFO ] Starting data processing...
[18:10:18][4976][INFO ] CUDA device ID: 0
[18:10:18][4976][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[18:10:18][4976][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[18:10:18][4976][INFO ] Version of installed CUDA driver: 9010
[18:10:18][4976][INFO ] Version of CUDA driver API used: 3020
[18:10:32][4976][INFO ] Number of generated templates to be used: 50000
[18:10:32][4976][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[18:10:32][4976][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[18:10:32][4976][INFO ] Seed for random number generator is -1030801420.
[18:10:37][4976][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[18:10:37][4976][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 994 MB (31 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 809 MB
[18:10:41][4976][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[18:10:41][4976][ERROR] Demodulation failed (error: 1006)!
[18:10:41][4976][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[18:31:28][9664][INFO ] Starting data processing...
[18:31:28][9664][INFO ] CUDA device ID: 0
[18:31:28][9664][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[18:31:28][9664][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[18:31:28][9664][INFO ] Version of installed CUDA driver: 9010
[18:31:28][9664][INFO ] Version of CUDA driver API used: 3020
[18:31:42][9664][INFO ] Number of generated templates to be used: 50000
[18:31:42][9664][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[18:31:42][9664][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[18:31:42][9664][INFO ] Seed for random number generator is -1030801420.
[18:31:47][9664][ERROR] Error allocating power spectrum device memory: 50332672 bytes (error: 2)
[18:31:47][9664][ERROR] Demodulation failed (error: 1006)!
[18:31:47][9664][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[18:47:15][6620][INFO ] Starting data processing...
[18:47:15][6620][INFO ] CUDA device ID: 0
[18:47:15][6620][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[18:47:15][6620][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[18:47:15][6620][INFO ] Version of installed CUDA driver: 9010
[18:47:15][6620][INFO ] Version of CUDA driver API used: 3020
[18:47:50][6620][INFO ] Number of generated templates to be used: 50000
[18:47:50][6620][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[18:47:50][6620][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[18:47:50][6620][INFO ] Seed for random number generator is -1030801420.
[18:47:55][6620][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[18:47:55][6620][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[18:47:55][6620][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[18:47:55][6620][ERROR] Demodulation failed (error: 1006)!
[18:47:55][6620][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[19:05:19][14792][INFO ] Starting data processing...
[19:05:19][14792][INFO ] CUDA device ID: 0
[19:05:19][14792][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:05:19][14792][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:05:19][14792][INFO ] Version of installed CUDA driver: 9010
[19:05:19][14792][INFO ] Version of CUDA driver API used: 3020
[19:05:52][14792][INFO ] Number of generated templates to be used: 50000
[19:05:52][14792][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[19:05:53][14792][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[19:05:53][14792][INFO ] Seed for random number generator is -1030801420.
[19:05:58][14792][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[19:05:58][14792][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[19:05:58][14792][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[19:05:58][14792][ERROR] Demodulation failed (error: 1006)!
[19:05:58][14792][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[19:20:58][14828][INFO ] Starting data processing...
[19:20:58][14828][INFO ] CUDA device ID: 0
[19:20:58][14828][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:20:58][14828][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:20:58][14828][INFO ] Version of installed CUDA driver: 9010
[19:20:58][14828][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[19:22:20][13672][INFO ] Starting data processing...
[19:22:20][13672][INFO ] CUDA device ID: 0
[19:22:20][13672][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:22:20][13672][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:22:20][13672][INFO ] Version of installed CUDA driver: 9010
[19:22:20][13672][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[19:23:09][11396][INFO ] Starting data processing...
[19:23:09][11396][INFO ] CUDA device ID: 0
[19:23:09][11396][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:23:09][11396][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:23:09][11396][INFO ] Version of installed CUDA driver: 9010
[19:23:09][11396][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[19:23:50][3780][INFO ] Starting data processing...
[19:23:50][3780][INFO ] CUDA device ID: 0
[19:23:50][3780][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:23:50][3780][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:23:50][3780][INFO ] Version of installed CUDA driver: 9010
[19:23:50][3780][INFO ] Version of CUDA driver API used: 3020
[19:24:04][3780][INFO ] Number of generated templates to be used: 50000
[19:24:04][3780][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[19:24:04][3780][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[19:24:04][3780][INFO ] Seed for random number generator is -1030801420.
[19:24:09][3780][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[19:24:09][3780][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[19:24:09][3780][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[19:24:09][3780][ERROR] Demodulation failed (error: 1006)!
[19:24:09][3780][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[19:39:20][10640][INFO ] Starting data processing...
[19:39:20][10640][INFO ] CUDA device ID: 0
[19:39:20][10640][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:39:20][10640][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:39:20][10640][INFO ] Version of installed CUDA driver: 9010
[19:39:20][10640][INFO ] Version of CUDA driver API used: 3020
[19:39:34][10640][INFO ] Number of generated templates to be used: 50000
[19:39:34][10640][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[19:39:34][10640][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[19:39:34][10640][INFO ] Seed for random number generator is -1030801420.
[19:39:39][10640][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[19:39:39][10640][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[19:39:39][10640][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[19:39:39][10640][ERROR] Demodulation failed (error: 1006)!
[19:39:39][10640][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[19:54:59][12092][INFO ] Starting data processing...
[19:54:59][12092][INFO ] CUDA device ID: 0
[19:54:59][12092][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:54:59][12092][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[19:54:59][12092][INFO ] Version of installed CUDA driver: 9010
[19:54:59][12092][INFO ] Version of CUDA driver API used: 3020
[19:55:13][12092][INFO ] Number of generated templates to be used: 50000
[19:55:13][12092][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[19:55:13][12092][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[19:55:14][12092][INFO ] Seed for random number generator is -1030801420.
[19:55:18][12092][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[19:55:18][12092][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[19:55:19][12092][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[19:55:19][12092][ERROR] Demodulation failed (error: 1006)!
[19:55:19][12092][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[20:10:39][8408][INFO ] Starting data processing...
[20:10:39][8408][INFO ] CUDA device ID: 0
[20:10:39][8408][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:10:39][8408][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:10:39][8408][INFO ] Version of installed CUDA driver: 9010
[20:10:39][8408][INFO ] Version of CUDA driver API used: 3020
[20:10:53][8408][INFO ] Number of generated templates to be used: 50000
[20:10:53][8408][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[20:10:53][8408][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[20:10:53][8408][INFO ] Seed for random number generator is -1030801420.
[20:10:58][8408][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[20:10:58][8408][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[20:10:58][8408][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[20:10:58][8408][ERROR] Demodulation failed (error: 1006)!
[20:10:58][8408][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[20:22:26][6220][INFO ] Starting data processing...
[20:22:26][6220][INFO ] CUDA device ID: 0
[20:22:26][6220][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:22:26][6220][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:22:26][6220][INFO ] Version of installed CUDA driver: 9010
[20:22:26][6220][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[20:25:17][14348][INFO ] Starting data processing...
[20:25:18][14348][INFO ] CUDA device ID: 0
[20:25:18][14348][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:25:18][14348][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:25:18][14348][INFO ] Version of installed CUDA driver: 9010
[20:25:18][14348][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[20:26:08][12132][INFO ] Starting data processing...
[20:26:08][12132][INFO ] CUDA device ID: 0
[20:26:08][12132][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:26:08][12132][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:26:08][12132][INFO ] Version of installed CUDA driver: 9010
[20:26:08][12132][INFO ] Version of CUDA driver API used: 3020
[20:26:42][12132][INFO ] Number of generated templates to be used: 50000
[20:26:42][12132][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[20:26:42][12132][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[20:26:42][12132][INFO ] Seed for random number generator is -1030801420.
[20:26:47][12132][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[20:26:47][12132][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[20:26:48][12132][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[20:26:48][12132][ERROR] Demodulation failed (error: 1006)!
[20:26:48][12132][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[20:41:57][13936][INFO ] Starting data processing...
[20:41:57][13936][INFO ] CUDA device ID: 0
[20:41:57][13936][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:41:57][13936][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:41:57][13936][INFO ] Version of installed CUDA driver: 9010
[20:41:57][13936][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[20:42:37][2964][INFO ] Starting data processing...
[20:42:37][2964][INFO ] CUDA device ID: 0
[20:42:37][2964][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:42:37][2964][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:42:37][2964][INFO ] Version of installed CUDA driver: 9010
[20:42:37][2964][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[20:43:48][1244][INFO ] Starting data processing...
[20:43:48][1244][INFO ] CUDA device ID: 0
[20:43:48][1244][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:43:48][1244][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:43:48][1244][INFO ] Version of installed CUDA driver: 9010
[20:43:48][1244][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[20:44:39][13340][INFO ] Starting data processing...
[20:44:40][13340][INFO ] CUDA device ID: 0
[20:44:40][13340][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:44:40][13340][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[20:44:40][13340][INFO ] Version of installed CUDA driver: 9010
[20:44:40][13340][INFO ] Version of CUDA driver API used: 3020
[20:45:13][13340][INFO ] Number of generated templates to be used: 50000
[20:45:13][13340][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[20:45:14][13340][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[20:45:14][13340][INFO ] Seed for random number generator is -1030801420.
[20:45:39][13340][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[20:45:39][13340][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 1025 MB (0 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 840 MB
[20:45:39][13340][ERROR] Couldn't allocate 37640 bytes of CUDA HS summing memory (error: 2)!
[20:45:39][13340][ERROR] Demodulation failed (error: 1006)!
[20:45:39][13340][WARN ] Sorry, at the moment your system doesn't have enough free CPU/GPU memory to run this task!
------> Returning control to BOINC, delaying next attempt for at least 15 minutes...
------> If this problem persists you should consider aborting this task...
Activated exception handling...
[21:26:02][16248][INFO ] Starting data processing...
[21:26:02][16248][INFO ] CUDA device ID: 0
[21:26:02][16248][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[21:26:02][16248][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[21:26:02][16248][INFO ] Version of installed CUDA driver: 9010
[21:26:02][16248][INFO ] Version of CUDA driver API used: 3020
Activated exception handling...
[01:09:16][9736][INFO ] Starting data processing...
[01:09:16][9736][INFO ] CUDA device ID: 0
[01:09:17][9736][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 185 MB (840 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[01:09:17][9736][INFO ] Using CUDA device #0 "Quadro K600" (192 CUDA cores / 336.19 GFLOPS)
[01:09:17][9736][INFO ] Version of installed CUDA driver: 9010
[01:09:17][9736][INFO ] Version of CUDA driver API used: 3020
[01:09:31][9736][INFO ] Number of generated templates to be used: 50000
[01:09:31][9736][INFO ] Checkpoint file unavailable: Ter5_1_dns_cfbf00113_segment_6_dms_200_181.cpt (No error).
------> Starting from scratch...
[01:09:31][9736][INFO ] Header contents:
------> Original WAPP file: /atlas/data/TRAPUM_GC/Terzan5/epoch1/30min_segments/dedispersed_files/cfbf00113/Ter5_1_dns_cfbf00113_segment_6_dms_200_DM248.10
------> Sample time in microseconds: 153.121
------> Observation time in seconds: 2568.9524
------> Time stamp (MJD): 59097.776207087605
------> Number of samples/record: 0
------> Center freq in MHz: 857.5673828
------> Channel band in MHz: 3.34375
------> Number of channels/record: 256
------> Nifs: 1
------> RA (J2000): 174804.33
------> DEC (J2000): -244749.700001
------> Galactic l: 0
------> Galactic b: 0
------> Name: J1748-2446N
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 16777216
------> Trial dispersion measure: 248.1 cm^-3 pc
------> Scale factor: 0.991492
[01:09:31][9736][INFO ] Seed for random number generator is -1030801420.
[01:09:36][9736][INFO ] Derived global search parameters:
------> f_A probability = 0.04
------> single bin prob(P_noise > P_thr) = 3.24424e-009
------> thr1 = 19.5464
------> thr2 = 22.7124
------> thr4 = 27.844
------> thr8 = 36.3869
------> thr16 = 50.9469
[01:09:36][9736][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 597 MB (428 MB free / 1025 MB total) -> Used by this application (assuming a single GPU task): 412 MB
[01:10:29][9736][INFO ] Checkpoint committed!
[01:11:29][9736][INFO ] Checkpoint committed!
[01:12:30][9736][INFO ] Checkpoint committed!
[01:13:29][9736][INFO ] Checkpoint committed!
[01:14:30][9736][INFO ] Checkpoint committed!
[01:15:30][9736][INFO ] Checkpoint committed!
[01:16:30][9736][INFO ] Checkpoint committed!
[01:17:30][9736][INFO ] Checkpoint committed!
[01:18:31][9736][INFO ] Checkpoint committed!
[01:19:31][9736][INFO ] Checkpoint committed!
[01:20:31][9736][INFO ] Checkpoint committed!
[01:21:40][9736][INFO ] Checkpoint committed!
[01:22:41][9736][INFO ] Checkpoint committed!
[01:23:41][9736][INFO ] Checkpoint committed!
[01:24:41][9736][INFO ] Checkpoint committed!
[01:25:41][9736][INFO ] Checkpoint committed!
[01:26:42][9736][INFO ] Checkpoint committed!
[01:27:42][9736][INFO ] Checkpoint committed!
[01:28:42][9736][INFO ] Checkpoint committed!
[01:29:42][9736][INFO ] Checkpoint committed!
[01:30:43][9736][INFO ] Checkpoint committed!
[01:31:43][9736][INFO ] Checkpoint committed!
[01:32:43][9736][INFO ] Checkpoint committed!
[01:33:43][9736][INFO ] Checkpoint committed!
[01:34:44][9736][INFO ] Checkpoint committed!
[01:35:44][9736][INFO ] Checkpoint committed!
[01:36:44][9736][INFO ] Checkpoint committed!
[01:37:52][9736][INFO ] Checkpoint committed!
[01:38:52][9736][INFO ] Checkpoint committed!
[01:39:52][9736][INFO ] Checkpoint committed!
[01:40:52][9736][INFO ] Checkpoint committed!
[01:41:53][9736][INFO ] Checkpoint committed!
[01:42:53][9736][INFO ] Checkpoint committed!
[01:44:02][9736][INFO ] Checkpoint committed!
[01:45:02][9736][INFO ] Checkpoint committed!
[01:46:03][9736][INFO ] Checkpoint committed!
[01:47:03][9736][INFO ] Checkpoint committed!
[01:48:03][9736][INFO ] Checkpoint committed!
[01:49:03][9736][INFO ] Checkpoint committed!
[01:50:04][9736][INFO ] Checkpoint committed!
[01:51:04][9736][INFO ] Checkpoint committed!
[01:52:04][9736][INFO ] Checkpoint committed!
[01:53:05][9736][INFO ] Checkpoint committed!
[01:54:05][9736][INFO ] Checkpoint committed!
[01:55:05][9736][INFO ] Checkpoint committed!
[01:56:13][9736][INFO ] Checkpoint committed!
[01:57:13][9736][INFO ] Checkpoint committed!
[01:58:13][9736][INFO ] Checkpoint committed!
[01:59:13][9736][INFO ] Checkpoint committed!
[02:00:13][9736][INFO ] Checkpoint committed!
[02:01:14][9736][INFO ] Checkpoint committed!
[02:02:14][9736][INFO ] Checkpoint committed!
[02:03:14][9736][INFO ] Checkpoint committed!
[02:04:14][9736][INFO ] Checkpoint committed!
[02:05:15][9736][INFO ] Checkpoint committed!
[02:06:15][9736][INFO ] Checkpoint committed!
[02:07:15][9736][INFO ] Checkpoint committed!
[02:08:15][9736][INFO ] Checkpoint committed!
[02:09:16][9736][INFO ] Checkpoint committed!
[02:10:16][9736][INFO ] Checkpoint committed!
[02:11:16][9736][INFO ] Checkpoint committed!
[02:12:16][9736][INFO ] Checkpoint committed!
[02:13:17][9736][INFO ] Checkpoint committed!
[02:14:17][9736][INFO ] Checkpoint committed!
[02:15:17][9736][INFO ] Checkpoint committed!
[02:16:18][9736][INFO ] Checkpoint committed!
[02:17:18][9736][INFO ] Checkpoint committed!
[02:18:18][9736][INFO ] Checkpoint committed!
[02:19:18][9736][INFO ] Checkpoint committed!
[02:20:18][9736][INFO ] Checkpoint committed!
[02:21:19][9736][INFO ] Checkpoint committed!
[02:22:19][9736][INFO ] Checkpoint committed!
[02:23:19][9736][INFO ] Checkpoint committed!
[02:24:20][9736][INFO ] Checkpoint committed!
[02:25:20][9736][INFO ] Checkpoint committed!
[02:26:20][9736][INFO ] Checkpoint committed!
[02:27:20][9736][INFO ] Checkpoint committed!
[02:28:21][9736][INFO ] Checkpoint committed!
[02:29:21][9736][INFO ] Checkpoint committed!
[02:30:21][9736][INFO ] Checkpoint committed!
[02:31:21][9736][INFO ] Checkpoint committed!
[02:32:22][9736][INFO ] Checkpoint committed!
[02:33:22][9736][INFO ] Checkpoint committed!
[02:34:22][9736][INFO ] Checkpoint committed!
[02:35:22][9736][INFO ] Checkpoint committed!
[02:36:23][9736][INFO ] Checkpoint committed!
[02:37:23][9736][INFO ] Checkpoint committed!
[02:38:23][9736][INFO ] Checkpoint committed!
[02:39:27][9736][INFO ] Checkpoint committed!
[02:40:27][9736][INFO ] Checkpoint committed!
[02:41:27][9736][INFO ] Checkpoint committed!
[02:42:27][9736][INFO ] Checkpoint committed!
[02:43:27][9736][INFO ] Checkpoint committed!
[02:44:28][9736][INFO ] Checkpoint committed!
[02:45:37][9736][INFO ] Checkpoint committed!
[02:46:37][9736][INFO ] Checkpoint committed!
[02:47:38][9736][INFO ] Checkpoint committed!
[02:48:57][9736][INFO ] Checkpoint committed!
[02:49:57][9736][INFO ] Checkpoint committed!
[02:50:58][9736][INFO ] Checkpoint committed!
[02:51:58][9736][INFO ] Checkpoint committed!
[02:53:18][9736][INFO ] Checkpoint committed!
[02:54:17][9736][INFO ] Checkpoint committed!
[02:55:18][9736][INFO ] Checkpoint committed!
[02:56:18][9736][INFO ] Checkpoint committed!
[02:57:18][9736][INFO ] Checkpoint committed!
[02:58:18][9736][INFO ] Checkpoint committed!
[02:59:19][9736][INFO ] Checkpoint committed!
[03:00:19][9736][INFO ] Checkpoint committed!
[03:01:19][9736][INFO ] Checkpoint committed!
[03:02:19][9736][INFO ] Checkpoint committed!
[03:03:20][9736][INFO ] Checkpoint committed!
[03:04:20][9736][INFO ] Checkpoint committed!
[03:05:20][9736][INFO ] Checkpoint committed!
[03:06:20][9736][INFO ] Checkpoint committed!
[03:07:21][9736][INFO ] Checkpoint committed!
[03:08:21][9736][INFO ] Checkpoint committed!
[03:09:21][9736][INFO ] Checkpoint committed!
[03:10:21][9736][INFO ] Checkpoint committed!
[03:11:22][9736][INFO ] Checkpoint committed!
[03:12:22][9736][INFO ] Checkpoint committed!
[03:13:22][9736][INFO ] Checkpoint committed!
[03:14:23][9736][INFO ] Checkpoint committed!
[03:15:23][9736][INFO ] Checkpoint committed!
[03:16:23][9736][INFO ] Checkpoint committed!
[03:17:23][9736][INFO ] Checkpoint committed!
[03:18:24][9736][INFO ] Checkpoint committed!
[03:19:24][9736][INFO ] Checkpoint committed!
[03:20:24][9736][INFO ] Checkpoint committed!
[03:21:24][9736][INFO ] Checkpoint committed!
[03:22:25][9736][INFO ] Checkpoint committed!
[03:23:25][9736][INFO ] Checkpoint committed!
[03:24:25][9736][INFO ] Checkpoint committed!
[03:25:25][9736][INFO ] Checkpoint committed!
[03:26:26][9736][INFO ] Checkpoint committed!
[03:27:26][9736][INFO ] Checkpoint committed!
[03:28:26][9736][INFO ] Checkpoint committed!
[03:29:27][9736][INFO ] Checkpoint committed!
[03:30:27][9736][INFO ] Checkpoint committed!
[03:31:27][9736][INFO ] Checkpoint committed!
[03:32:27][9736][INFO ] Checkpoint committed!
[03:33:31][9736][INFO ] Checkpoint committed!
[03:34:31][9736][INFO ] Checkpoint committed!
[03:35:31][9736][INFO ] Checkpoint committed!
[03:36:32][9736][INFO ] Checkpoint committed!
[03:37:32][9736][INFO ] Checkpoint committed!
[03:38:32][9736][INFO ] Checkpoint committed!
[03:39:32][9736][INFO ] Checkpoint committed!
[03:40:33][9736][INFO ] Checkpoint committed!
[03:41:33][9736][INFO ] Checkpoint committed!
[03:42:33][9736][INFO ] Checkpoint committed!
[03:43:33][9736][INFO ] Checkpoint committed!
[03:44:52][9736][INFO ] Checkpoint committed!
[03:45:52][9736][INFO ] Checkpoint committed!
[03:46:52][9736][INFO ] Checkpoint committed!
[03:47:52][9736][INFO ] Checkpoint committed!
[03:49:12][9736][INFO ] Checkpoint committed!
[03:50:12][9736][INFO ] Checkpoint committed!
[03:51:12][9736][INFO ] Checkpoint committed!
[03:52:13][9736][INFO ] Checkpoint committed!
[03:53:13][9736][INFO ] Checkpoint committed!
[03:54:13][9736][INFO ] Checkpoint committed!
[03:55:13][9736][INFO ] Checkpoint committed!
[03:56:14][9736][INFO ] Checkpoint committed!
[03:57:14][9736][INFO ] Checkpoint committed!
[03:58:14][9736][INFO ] Checkpoint committed!
[03:59:15][9736][INFO ] Checkpoint committed!
[04:01:23][9736][INFO ] Checkpoint committed!
[04:02:22][9736][INFO ] Checkpoint committed!
[04:03:22][9736][INFO ] Checkpoint committed!
[04:04:23][9736][INFO ] Checkpoint committed!
[04:05:23][9736][INFO ] Checkpoint committed!
[04:06:23][9736][INFO ] Checkpoint committed!
[04:07:24][9736][INFO ] Checkpoint committed!
[04:08:24][9736][INFO ] Checkpoint committed!
[04:09:24][9736][INFO ] Checkpoint committed!
[04:10:24][9736][INFO ] Checkpoint committed!
[04:11:25][9736][INFO ] Checkpoint committed!
[04:12:25][9736][INFO ] Checkpoint committed!
[04:13:25][9736][INFO ] Checkpoint committed!
[04:14:26][9736][INFO ] Checkpoint committed!
[04:15:26][9736][INFO ] Checkpoint committed!
[04:16:26][9736][INFO ] Checkpoint committed!
[04:17:26][9736][INFO ] Checkpoint committed!
[04:18:27][9736][INFO ] Checkpoint committed!
[04:19:27][9736][INFO ] Checkpoint committed!
[04:20:27][9736][INFO ] Checkpoint committed!
[04:21:27][9736][INFO ] Checkpoint committed!
[04:22:28][9736][INFO ] Checkpoint committed!
[04:23:35][9736][INFO ] Checkpoint committed!
[04:24:34][9736][INFO ] Checkpoint committed!
[04:25:35][9736][INFO ] Checkpoint committed!
[04:26:35][9736][INFO ] Checkpoint committed!
[04:27:35][9736][INFO ] Checkpoint committed!
[04:28:55][9736][INFO ] Checkpoint committed!
[04:29:54][9736][INFO ] Checkpoint committed!
[04:30:55][9736][INFO ] Checkpoint committed!
[04:31:55][9736][INFO ] Checkpoint committed!
[04:32:55][9736][INFO ] Checkpoint committed!
[04:33:56][9736][INFO ] Checkpoint committed!
[04:34:56][9736][INFO ] Checkpoint committed!
[04:35:56][9736][INFO ] Checkpoint committed!
[04:36:56][9736][INFO ] Checkpoint committed!
[04:38:16][9736][INFO ] Checkpoint committed!
[04:39:16][9736][INFO ] Checkpoint committed!
[04:40:16][9736][INFO ] Checkpoint committed!
[04:41:17][9736][INFO ] Checkpoint committed!
[04:42:17][9736][INFO ] Checkpoint committed!
[04:43:17][9736][INFO ] Checkpoint committed!
[04:44:17][9736][INFO ] Checkpoint committed!
[04:45:26][9736][INFO ] Checkpoint committed!
[04:46:26][9736][INFO ] Checkpoint committed!
[04:47:26][9736][INFO ] Checkpoint committed!
[04:48:26][9736][INFO ] Checkpoint committed!
[04:49:27][9736][INFO ] Checkpoint committed!
[04:50:27][9736][INFO ] Checkpoint committed!
[04:51:27][9736][INFO ] Checkpoint committed!
[04:52:47][9736][INFO ] Checkpoint committed!
[04:53:47][9736][INFO ] Checkpoint committed!
[04:54:47][9736][INFO ] Checkpoint committed!
[04:55:47][9736][INFO ] Checkpoint committed!
[04:56:48][9736][INFO ] Checkpoint committed!
[04:57:48][9736][INFO ] Checkpoint committed!
[04:58:48][9736][INFO ] Checkpoint committed!
[04:59:48][9736][INFO ] Checkpoint committed!
[05:00:49][9736][INFO ] Checkpoint committed!
[05:01:49][9736][INFO ] Checkpoint committed!
[05:02:49][9736][INFO ] Checkpoint committed!
[05:03:49][9736][INFO ] Checkpoint committed!
[05:04:50][9736][INFO ] Checkpoint committed!
[05:05:50][9736][INFO ] Checkpoint committed!
[05:06:58][9736][INFO ] Checkpoint committed!
[05:07:57][9736][INFO ] Checkpoint committed!
[05:08:58][9736][INFO ] Checkpoint committed!
[05:09:58][9736][INFO ] Checkpoint committed!
[05:10:58][9736][INFO ] Checkpoint committed!
[05:11:58][9736][INFO ] Checkpoint committed!
[05:12:59][9736][INFO ] Checkpoint committed!
[05:13:59][9736][INFO ] Checkpoint committed!
[05:14:59][9736][INFO ] Checkpoint committed!
[05:16:00][9736][INFO ] Checkpoint committed!
[05:17:00][9736][INFO ] Checkpoint committed!
[05:18:09][9736][INFO ] Checkpoint committed!
[05:19:09][9736][INFO ] Checkpoint committed!
[05:20:09][9736][INFO ] Checkpoint committed!
[05:21:09][9736][INFO ] Checkpoint committed!
[05:22:09][9736][INFO ] Checkpoint committed!
[05:23:09][9736][INFO ] Checkpoint committed!
[05:24:10][9736][INFO ] Checkpoint committed!
[05:25:10][9736][INFO ] Checkpoint committed!
[05:26:10][9736][INFO ] Checkpoint committed!
[05:27:10][9736][INFO ] Checkpoint committed!
[05:28:11][9736][INFO ] Checkpoint committed!
[05:29:11][9736][INFO ] Checkpoint committed!
[05:30:11][9736][INFO ] Checkpoint committed!
[05:31:11][9736][INFO ] Checkpoint committed!
[05:32:12][9736][INFO ] Checkpoint committed!
[05:33:12][9736][INFO ] Checkpoint committed!
[05:34:12][9736][INFO ] Checkpoint committed!
[05:35:12][9736][INFO ] Checkpoint committed!
[05:36:13][9736][INFO ] Checkpoint committed!
[05:37:13][9736][INFO ] Checkpoint committed!
[05:38:13][9736][INFO ] Checkpoint committed!
[05:39:14][9736][INFO ] Checkpoint committed!
[05:40:14][9736][INFO ] Checkpoint committed!
[05:41:14][9736][INFO ] Checkpoint committed!
[05:42:14][9736][INFO ] Checkpoint committed!
[05:43:14][9736][INFO ] Checkpoint committed!
[05:44:15][9736][INFO ] Checkpoint committed!
[05:45:15][9736][INFO ] Checkpoint committed!
[05:46:15][9736][INFO ] Checkpoint committed!
[05:47:16][9736][INFO ] Checkpoint committed!
[05:48:16][9736][INFO ] Checkpoint committed!
[05:49:16][9736][INFO ] Checkpoint committed!
[05:50:21][9736][INFO ] Checkpoint committed!
[05:51:21][9736][INFO ] Checkpoint committed!
[05:52:21][9736][INFO ] Checkpoint committed!
[05:53:21][9736][INFO ] Checkpoint committed!
[05:54:21][9736][INFO ] Checkpoint committed!
[05:55:22][9736][INFO ] Checkpoint committed!
[05:56:22][9736][INFO ] Checkpoint committed!
[05:57:22][9736][INFO ] Checkpoint committed!
[05:58:22][9736][INFO ] Checkpoint committed!
[05:59:23][9736][INFO ] Checkpoint committed!
[06:00:23][9736][INFO ] Checkpoint committed!
[06:01:23][9736][INFO ] Checkpoint committed!
[06:02:24][9736][INFO ] Checkpoint committed!
[06:03:24][9736][INFO ] Checkpoint committed!
[06:03:37][9736][INFO ] Statistics: count dirty SumSpec pages 241504 (not checkpointed), Page Size 1024, fundamental_idx_hi-window_2: 1926214
[06:03:37][9736][INFO ] Data processing finished successfully!
06:03:37 (9736): called boinc_finish(0)

</stderr_txt>
]]>