There appears to be a problem with a few input files of this search. A lot of tasks crash immediately while trying to read these (no computing time wasted, though). I suspended this application (no more such tasks sent) while investigating.
The problem has been diagnosed, and for now no more workunits that use these input files will be produced. We had to cancel a few workunits, but these would have ended in client errors anyway. Apart from that, the run is resumed.
I have 2 tasks that will not completely upload. They get to 100%, then a long pause, then an error -- Upload: pending (project backoff: ####)
Here is my log (pasted below).
It has been several days now, and no new projects will download (I run both CPU and GPU tasks) because of the stuck uploads I believe.
Should I abort them? I have another machine running tasks for the Gamma Ray pulsar search and it is successfully communicating. (which leads me to believe my firewall is not the cause)
Any advice would be helpful.
Thanks.
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:15 AM | Einstein@Home | Started upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:15 AM | Einstein@Home | Started upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [serially]
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Trying 130.75.116.34...
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Hostname was found in DNS cache
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Trying 130.75.116.34...
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#56)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept: */*
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Length: 299
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: We are completely uploaded and fine
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#57)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept: */*
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Length: 299
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: We are completely uploaded and fine
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 200 OK
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Server: nginx
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Date: Fri, 08 Mar 2019 12:11:17 GMT
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Type: text/plain
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Transfer-Encoding: chunked
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Connection: keep-alive
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: 5d
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <status>0</status>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <file_size>0</file_size>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: </data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: 0
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | | [http_xfer] [ID#117] HTTP: wrote 93 bytes
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Connection #57 to host einstein4.aei.uni-hannover.de left intact
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 200 OK
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Server: nginx
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Date: Fri, 08 Mar 2019 12:11:17 GMT
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Type: text/plain
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Transfer-Encoding: chunked
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Connection: keep-alive
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: 5d
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <status>0</status>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <file_size>0</file_size>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: </data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: 0
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | | [http_xfer] [ID#116] HTTP: wrote 93 bytes
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Connection #56 to host einstein4.aei.uni-hannover.de left intact
3/8/2019 6:11:16 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:16 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [can pipeline]
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Re-using existing connection! (#56) with host einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#56)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept: */*
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Length: 2349767
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Expect: 100-continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server:
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [can pipeline]
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Re-using existing connection! (#57) with host einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#57)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept: */*
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Length: 2340444
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Expect: 100-continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server:
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 100 Continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 100 Continue
3/8/2019 6:11:17 AM | Einstein@Home | [http] [ID#116] Info: We are completely uploaded and fine
3/8/2019 6:11:17 AM | Einstein@Home | [http] [ID#117] Info: We are completely uploaded and fine
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 504 Timeout while reading response from Server
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Date: Fri, 08 Mar 2019 12:12:01 GMT
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Cache-Control: no-cache
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Pragma: no-cache
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Type: text/html; charset="UTF-8"
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Length: 0
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Connection: close
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Info: Closing connection 56
3/8/2019 6:12:18 AM | Einstein@Home | Temporarily failed upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0: transient HTTP error
3/8/2019 6:12:18 AM | Einstein@Home | Backing off 03:27:45 on upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 504 Timeout while reading response from Server
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Date: Fri, 08 Mar 2019 12:12:02 GMT
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Cache-Control: no-cache
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Pragma: no-cache
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Type: text/html; charset="UTF-8"
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Length: 0
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Connection: close
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Info: Closing connection 57
3/8/2019 6:12:19 AM | Einstein@Home | Temporarily failed upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1: transient HTTP error
3/8/2019 6:12:19 AM | Einstein@Home | Backing off 03:06:49 on upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1
The batch of data for O1OD1 is running low. Server status says 9.7 left days - but that includes tasks issued but not returned - so they're probably going to run out within the week. Does it look good for the next batch being ready in time, or will we be looking at a repeat of the extended GW task outages from the last two years.
There are a couple of workunits that we skipped in the current run due to input files that were broken (in pre-processing). We fixed this, and I'm currently preparing these to be re-ran. I expect this to give us "work" for about another week, though.
For scientific evaluation of the results of O1OD1 we will need another, shorter run (with "injected", fake signals), which we are currently preparing. We keep finding irregularities and inconsistencies, though, so it's not yet clear how long this will take.
We will probably finish the previously interrupted O2AS run next, this is something that won't need any more time for preparation, and should give us time enough to finish preparations for the "injection" run.
All in all, the next months of gravitational wave data analysis should be secured.
We decided to do an intermediate "Engineering run" ("O1OD1E"), primarily to validate a GPU version of the O1OD1 application. Unfortunately we were slowed down in setting this up by a number of unrelated but urgent problems, but now we're basically ready. Sorry for the slow progress on that, particularly for the slow validation.
Currently there are CPU app versions for basically every platform, and GPU versions will be added when available and incrementally, as these have undergone only very limited internal testing. GPU app versions are marked "beta test" versions, so in order to get such "work", you will need to have Beta test applications enabled in your project preferences.
The applications use OpenCL (1.2). Currently we are sipping app versions only for NVidia cards on Linux and Mac OSX. There are indications that the current Windows App version is not "portable", i.e. doesn't run on all Windows installation, so we disabled it for the moment. More App versions for other platforms (Windows) and GPUs (AMD, Intel) as we gain confidence in the apps that have been published.
The Beta status of the GPU app versions means that every result of a GPU App version needs to be "validated" by that of a CPU version. This is intentional, actually it's the main purpose of that run to find out how the results compare.
<core_client_version>7.6.31</core_client_version>
<![CDATA[
<message>
process exited with code 127 (0x7f, -129)
</message>
<stderr_txt>
../../projects/einstein.phys.uwm.edu/einstein_O1OD1E_0.05_x86_64-pc-linux-gnu__GW-opencl-nvidia-V1: relocation error: ../../projects/einstein.phys.uwm.edu/einstein_O1OD1E_0.05_x86_64-pc-linux-gnu__GW-opencl-nvidia-V1: symbol __get_cpu_features, version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference
Same host as yesterday under Win7 Pro, no work for my 1050Ti:
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work sent
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work is available for Gravitational Wave All-sky search on LIGO O1 Open Data
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work is available for Gravitational Wave Engineering run on LIGO O1 Open Data
2019-03-28 13:10:20.9519 [PID=23397] Sending reply to [HOST#12247194]: 0 results, delay req 60.00
Thanks for reporting. Should
)
Thanks for reporting. Should be fixed for newly created tasks. Sorry for the error.
BM
There appears to be a problem
)
There appears to be a problem with a few input files of this search. A lot of tasks crash immediately while trying to read these (no computing time wasted, though). I suspended this application (no more such tasks sent) while investigating.
BM
The problem has been
)
The problem has been diagnosed, and for now no more workunits that use these input files will be produced. We had to cancel a few workunits, but these would have ended in client errors anyway. Apart from that, the run is resumed.
BM
Hello, I have 2 tasks that
)
Hello,
I have 2 tasks that will not completely upload. They get to 100%, then a long pause, then an error -- Upload: pending (project backoff: ####)
Here is my log (pasted below).
It has been several days now, and no new projects will download (I run both CPU and GPU tasks) because of the stuck uploads I believe.
Should I abort them? I have another machine running tasks for the Gamma Ray pulsar search and it is successfully communicating. (which leads me to believe my firewall is not the cause)
Any advice would be helpful.
Thanks.
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:15 AM | Einstein@Home | Started upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle 'C:\Program Files\BOINC\ca-bundle.crt'
3/8/2019 6:11:15 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:15 AM | Einstein@Home | Started upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [serially]
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Trying 130.75.116.34...
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Hostname was found in DNS cache
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Trying 130.75.116.34...
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#56)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept: */*
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Length: 299
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Sent header to server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: We are completely uploaded and fine
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#57)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept: */*
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Length: 299
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Sent header to server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: We are completely uploaded and fine
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 200 OK
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Server: nginx
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Date: Fri, 08 Mar 2019 12:11:17 GMT
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Type: text/plain
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Transfer-Encoding: chunked
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: Connection: keep-alive
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: 5d
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <status>0</status>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: <file_size>0</file_size>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: </data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server: 0
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:11:15 AM | | [http_xfer] [ID#117] HTTP: wrote 93 bytes
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#117] Info: Connection #57 to host einstein4.aei.uni-hannover.de left intact
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 200 OK
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Server: nginx
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Date: Fri, 08 Mar 2019 12:11:17 GMT
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Type: text/plain
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Transfer-Encoding: chunked
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: Connection: keep-alive
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: 5d
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <status>0</status>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: <file_size>0</file_size>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: </data_server_reply>
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server: 0
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:11:15 AM | | [http_xfer] [ID#116] HTTP: wrote 93 bytes
3/8/2019 6:11:15 AM | Einstein@Home | [http] [ID#116] Info: Connection #56 to host einstein4.aei.uni-hannover.de left intact
3/8/2019 6:11:16 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:16 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [can pipeline]
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Re-using existing connection! (#56) with host einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#56)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept: */*
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Content-Length: 2349767
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server: Expect: 100-continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Sent header to server:
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Found bundle for host einstein4.aei.uni-hannover.de: 0x393c080 [can pipeline]
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Re-using existing connection! (#57) with host einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Info: Connected to einstein4.aei.uni-hannover.de (130.75.116.34) port 80 (#57)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: POST /EinsteinAtHome/cgi-bin/file_upload_handler_large HTTP/1.1
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Host: einstein4.aei.uni-hannover.de
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.14.2)
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept: */*
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Encoding: deflate, gzip
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Type: application/x-www-form-urlencoded
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Accept-Language: en_US
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Content-Length: 2340444
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server: Expect: 100-continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Sent header to server:
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 100 Continue
3/8/2019 6:11:16 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 100 Continue
3/8/2019 6:11:17 AM | Einstein@Home | [http] [ID#116] Info: We are completely uploaded and fine
3/8/2019 6:11:17 AM | Einstein@Home | [http] [ID#117] Info: We are completely uploaded and fine
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: HTTP/1.1 504 Timeout while reading response from Server
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Date: Fri, 08 Mar 2019 12:12:01 GMT
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Cache-Control: no-cache
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Pragma: no-cache
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Type: text/html; charset="UTF-8"
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Content-Length: 0
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server: Connection: close
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Received header from server:
3/8/2019 6:12:17 AM | Einstein@Home | [http] [ID#116] Info: Closing connection 56
3/8/2019 6:12:18 AM | Einstein@Home | Temporarily failed upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0: transient HTTP error
3/8/2019 6:12:18 AM | Einstein@Home | Backing off 03:27:45 on upload of h1_0415.15_O1C02Cl2In0__O1OD1_415.35Hz_973_0_0
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: HTTP/1.1 504 Timeout while reading response from Server
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Date: Fri, 08 Mar 2019 12:12:02 GMT
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Cache-Control: no-cache
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Pragma: no-cache
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Type: text/html; charset="UTF-8"
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Content-Length: 0
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Via: HTTP/1.1 forward.http.proxy:3128
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server: Connection: close
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Received header from server:
3/8/2019 6:12:18 AM | Einstein@Home | [http] [ID#117] Info: Closing connection 57
3/8/2019 6:12:19 AM | Einstein@Home | Temporarily failed upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1: transient HTTP error
3/8/2019 6:12:19 AM | Einstein@Home | Backing off 03:06:49 on upload of h1_0459.95_O1C02Cl2In0__O1OD1_460.20Hz_516_0_1
The batch of data for O1OD1
)
The batch of data for O1OD1 is running low. Server status says 9.7 left days - but that includes tasks issued but not returned - so they're probably going to run out within the week. Does it look good for the next batch being ready in time, or will we be looking at a repeat of the extended GW task outages from the last two years.
There are a couple of
)
There are a couple of workunits that we skipped in the current run due to input files that were broken (in pre-processing). We fixed this, and I'm currently preparing these to be re-ran. I expect this to give us "work" for about another week, though.
For scientific evaluation of the results of O1OD1 we will need another, shorter run (with "injected", fake signals), which we are currently preparing. We keep finding irregularities and inconsistencies, though, so it's not yet clear how long this will take.
We will probably finish the previously interrupted O2AS run next, this is something that won't need any more time for preparation, and should give us time enough to finish preparations for the "injection" run.
All in all, the next months of gravitational wave data analysis should be secured.
BM
We decided to do an
)
We decided to do an intermediate "Engineering run" ("O1OD1E"), primarily to validate a GPU version of the O1OD1 application. Unfortunately we were slowed down in setting this up by a number of unrelated but urgent problems, but now we're basically ready. Sorry for the slow progress on that, particularly for the slow validation.
Currently there are CPU app versions for basically every platform, and GPU versions will be added when available and incrementally, as these have undergone only very limited internal testing. GPU app versions are marked "beta test" versions, so in order to get such "work", you will need to have Beta test applications enabled in your project preferences.
The applications use OpenCL (1.2). Currently we are sipping app versions only for NVidia cards on Linux and Mac OSX. There are indications that the current Windows App version is not "portable", i.e. doesn't run on all Windows installation, so we disabled it for the moment. More App versions for other platforms (Windows) and GPUs (AMD, Intel) as we gain confidence in the apps that have been published.
The Beta status of the GPU app versions means that every result of a GPU App version needs to be "validated" by that of a CPU version. This is intentional, actually it's the main purpose of that run to find out how the results compare.
BM
Linux Mint 18 Sarah, NVIDIA
)
Linux Mint 18 Sarah, NVIDIA Driver 384.130:
I tried this GPU-task, but no luck:
</stderr_txt>
]]>
There is a new attempt on a
)
There is a new attempt on a Windows version (0.05). Let's see how this performs.
BM
2019-03-28 13:10:20.9518
)
Same host as yesterday under Win7 Pro, no work for my 1050Ti:
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work sent
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work is available for Gravitational Wave All-sky search on LIGO O1 Open Data
2019-03-28 13:10:20.9518 [PID=23397] [debug] [HOST#12247194] MSG(high) No work is available for Gravitational Wave Engineering run on LIGO O1 Open Data
2019-03-28 13:10:20.9519 [PID=23397] Sending reply to [HOST#12247194]: 0 results, delay req 60.00