backup project: resource share problem, log entries
28 Jul 2005 9:29:08 UTC
Topic 189622
(moderation:
This posting belongs with this original thread - I have posted this in a separate thread, as the preformatted log entries make the rest of the page unreadable. Please put all replies on the original thread unless your posting also contains log entries.
Machine is a single processor machine with the following projects and resource shares.
Orbit, 9900 (main project, known to have no work available)
LHC, 99 (first level backup project)
Predictor, 2 (intended to be a second level backup project, ie hopefully runs only when neither Orbit nor LHC have work)
Found the machine with completed work for both backup projects, but making no attempt to get new work. Between the first and last entry shown here, nothing has been deleted, you have the complete log. I have added annotations in italic interspesed with the logs: obviously my comments are fallible, and experts may interpret the log entries differently.
[pre]
28/07/2005 08:34:34|LHC@home|Deferring communication with project for 1 days, 7 hours, 29 minutes, and 16 seconds
28/07/2005 08:34:34|ProteinPredictorAtHome|Deferring communication with project for 1 days, 15 hours, 37 minutes, and 20 seconds
28/07/2005 08:34:34||Insufficient work; requesting more
28/07/2005 08:34:34|orbit@home|Deferring communication with project for 1 hours, 42 minutes, and 11 seconds
[/pre]
so, although all local work is complete, nothing more will get done for over a day.
at this point I clicked update on LHC and then Predictor immediately after, as I did not want the machine idle for so long:
[pre]
28/07/2005 08:35:22||request_reschedule_cpus: project op
28/07/2005 08:35:22|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
28/07/2005 08:35:22|LHC@home|Requesting 60480 seconds of work, returning 1 results
28/07/2005 08:35:24|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi succeeded
28/07/2005 08:35:25|LHC@home|Started download of wjun1D_v6s4hvnom_mqx__14__64.315_59.325__8_10__6__10_1_sixvf_boinc18084.zip
28/07/2005 08:35:27|LHC@home|Finished download of wjun1D_v6s4hvnom_mqx__14__64.315_59.325__8_10__6__10_1_sixvf_boinc18084.zip
28/07/2005 08:35:27|LHC@home|Throughput 22793 bytes/sec
28/07/2005 08:35:27||request_reschedule_cpus: files downloaded
28/07/2005 08:35:27|LHC@home|Starting result wjun1D_v6s4hvnom_mqx__14__64.315_59.325__8_10__6__10_1_sixvf_boinc18084_2 using sixtrack version 4.67
[/pre]
There now follows a little double take as the client first decides the machine is over committed and then changes its mind. This is not a problem, but can look confusing when reading the logs
[pre]
28/07/2005 08:35:38||Suspending work fetch because computer is overcommitted.
28/07/2005 08:35:38||Using earliest-deadline-first scheduling because computer is overcommitted.
28/07/2005 08:35:44||request_reschedule_cpus: project op
28/07/2005 08:35:44||Allowing work fetch again.
28/07/2005 08:35:44||Resuming round-robin CPU scheduling.
[/pre]
and now the client gets around to obeying my update request for Predictor
[pre]
28/07/2005 08:35:45|ProteinPredictorAtHome|Sending scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi
28/07/2005 08:35:45|ProteinPredictorAtHome|Requesting 60480 seconds of work, returning 1 results
28/07/2005 08:35:52|ProteinPredictorAtHome|Scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi succeeded
28/07/2005 08:35:52|ProteinPredictorAtHome|Message from server: No work sent
28/07/2005 08:35:52|ProteinPredictorAtHome|Message from server: (there was work but it was committed to other platforms)
[/pre]
so far, so good, Predictor did not have any work for my platform, this is not part of the problem I am reporting.
But look what happens now, as the client notices it is already 'close to the end' of the single LHC wu it downloaded. (est time to completion, 11 hours, connect interval = 0.7 days, which is more than 11 hours)
[pre]
28/07/2005 08:36:04|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
28/07/2005 08:36:04|LHC@home|Requesting 428 seconds of work, returning 0 results
28/07/2005 08:36:05|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi succeeded
28/07/2005 08:36:05|LHC@home|Message from server: No work sent
28/07/2005 08:36:05|LHC@home|Message from server: (won't finish in time) Computer on 99.7% of time, BOINC on 100.0% of that, this project gets 1.0% of that
28/07/2005 08:36:05|LHC@home|No work from project
28/07/2005 08:36:06|LHC@home|Deferring communication with project for 1 days, 23 hours, 59 minutes, and 58 seconds
[/pre]
so now, with half a day's work on board, my client is asked not to go back to LHC for two days...
Nothing happens for an hour, then the next log entry, with no further input from me, is
[pre]
28/07/2005 09:34:35||May run out of work in 0.70 days; requesting more
28/07/2005 09:34:35|orbit@home|Deferring communication with project for 42 minutes and 11 seconds
28/07/2005 09:34:35|ProteinPredictorAtHome|Deferring communication with project for 1 minutes and 16 seconds
28/07/2005 09:35:56|ProteinPredictorAtHome|Sending scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi
28/07/2005 09:35:56|ProteinPredictorAtHome|Requesting 60480 seconds of work, returning 0 results
28/07/2005 09:36:02|ProteinPredictorAtHome|Scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi succeeded
28/07/2005 09:36:03|ProteinPredictorAtHome|Sending scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi
28/07/2005 09:36:03|ProteinPredictorAtHome|Requesting 5140 seconds of work, returning 0 results
28/07/2005 09:36:03|ProteinPredictorAtHome|Started download of h0010A_1_192303.ini
28/07/2005 09:36:03|ProteinPredictorAtHome|Started download of h0010A_1_192303.inp
28/07/2005 09:36:04|ProteinPredictorAtHome|Finished download of h0010A_1_192303.ini
28/07/2005 09:36:04|ProteinPredictorAtHome|Throughput 3064 bytes/sec
28/07/2005 09:36:04|ProteinPredictorAtHome|Finished download of h0010A_1_192303.inp
28/07/2005 09:36:04|ProteinPredictorAtHome|Throughput 329 bytes/sec
28/07/2005 09:36:04|ProteinPredictorAtHome|Started download of h0010A_1_192303.seq
28/07/2005 09:36:04|ProteinPredictorAtHome|Started download of h0010A_1_192303.res[/pre]
so now we have a single Predictor wu to go with our single LHC wu. This won't be enough so the client immediately goes back to ask for another Predictor wu:
[pre]
28/07/2005 09:36:05|ProteinPredictorAtHome|Scheduler request to http://predictor.scripps.edu/predictor_cgi/cgi succeeded
28/07/2005 09:36:05|ProteinPredictorAtHome|Message from server: No work sent
28/07/2005 09:36:05|ProteinPredictorAtHome|Message from server: (won't finish in time) Computer on 99.7% of time, BOINC on 100.0% of that, this project gets 0.0% of that
28/07/2005 09:36:05|ProteinPredictorAtHome|No work from project
[/pre]
but of course the download for the one wu is still in progress, that continues OK
[pre]
28/07/2005 09:36:05|ProteinPredictorAtHome|Finished download of h0010A_1_192303.seq
28/07/2005 09:36:05|ProteinPredictorAtHome|Throughput 3829 bytes/sec
28/07/2005 09:36:05|ProteinPredictorAtHome|Finished download of h0010A_1_192303.res
28/07/2005 09:36:05|ProteinPredictorAtHome|Throughput 7 bytes/sec
28/07/2005 09:36:05||request_reschedule_cpus: files downloaded
28/07/2005 09:36:05|LHC@home|Pausing result wjun1D_v6s4hvnom_mqx__14__64.315_59.325__8_10__6__10_1_sixvf_boinc18084_2 (left in memory)
28/07/2005 09:36:05|ProteinPredictorAtHome|Starting result h0010A_1_192303_3 using mfoldB125 version 4.28
28/07/2005 09:36:06|LHC@home|Deferring communication with project for 1 days, 22 hours, 59 minutes, and 58 seconds
28/07/2005 09:36:06|ProteinPredictorAtHome|Deferring communication with project for 1 days, 23 hours, 59 minutes, and 58 seconds
[/pre]
so we leave these extracts from the log with the machine holding work for Predictor (about 2 to 3 hours worth) and for LHC (about 10 hours worth) and no contact with either project for nearly two days
This seems to me to be a loop: whenever the project does download new work, it then refuses the next request for more work (as the scheduler thinks it will not finish existing work in time) and locks out for longer than the work in hand will actually take to complete in the abesnce of work appearing from Orbit. The machine will be utilised just over 1/4 of the time till work is available from Orbit.
Please let me know if it would also be useful to post extracts from the log on the 2 cpu machine, or is this extract sufficient to make the issue clear?
~~gravywavy
backup project: resource share problem, log entries
)
This posting belongs with this posting within this original thread - I have posted this in a separate thread, as the preformatted log entries make the rest of the page unreadable.
Please put all replies on the original thread unless your posting also contains log entries
This log shows:
Single cpu machine running LHC and Einstein, Respource shares LHC 9999, Einstein 1. That is E@h is taking the role of backup project. As both projects are recently attached there is a wu from each project on hand when the connection to LHC is deliberately broken for test purposes (this break is done locally, within my LAN, and does not affect the project itself).
When LHC first tries to request new work the normal exponential backoff starts. When it tries to report exponential backoff restarts at first, growing to a backoff of between 3 and 4 hours by around 14:00:00, but then the exponential backoff collpses and it goes into a mode where it retries every minute or so. Why this is bad is described in the original thread.
[pre]
30/07/2005 22:43:00||Starting BOINC client version 4.45 for windows_intelx86
30/07/2005 22:43:00||Data directory: C:Program FilesBOINC
30/07/2005 22:43:00|Einstein@Home|Computer ID: 382673; location: work; project prefs: work
30/07/2005 22:43:00|LHC@home|Computer ID: 44884; location: work; project prefs: work
30/07/2005 22:43:00||General prefs: from Einstein@Home (last modified 2005-07-30 22:40:42)
30/07/2005 22:43:00||General prefs: using separate prefs for work
30/07/2005 22:43:00||Remote control not allowed; using loopback address
30/07/2005 22:43:00|Einstein@Home|Deferring computation for result w1_0876.5__0876.9_0.1_T00_S4hA_2
30/07/2005 22:43:00|LHC@home|Resuming computation for result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 using sixtrack version 4.67
30/07/2005 22:48:44||request_reschedule_cpus: project op
30/07/2005 22:48:45|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
30/07/2005 22:48:45|LHC@home|Requesting 0 seconds of work, returning 0 results
30/07/2005 22:48:46|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi succeeded
31/07/2005 01:48:44|Einstein@Home|Restarting result w1_0876.5__0876.9_0.1_T00_S4hA_2 using einstein version 4.79
31/07/2005 01:48:44|LHC@home|Pausing result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 (removed from memory)
31/07/2005 01:48:45||request_reschedule_cpus: process exited
31/07/2005 01:48:45|Einstein@Home|Pausing result w1_0876.5__0876.9_0.1_T00_S4hA_2 (removed from memory)
31/07/2005 01:48:45|LHC@home|Restarting result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 using sixtrack version 4.67
31/07/2005 03:48:45|Einstein@Home|Resuming result w1_0876.5__0876.9_0.1_T00_S4hA_2 using einstein version 4.79
31/07/2005 03:48:45|LHC@home|Pausing result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 (removed from memory)
31/07/2005 03:48:47||request_reschedule_cpus: process exited
31/07/2005 03:48:47|Einstein@Home|Pausing result w1_0876.5__0876.9_0.1_T00_S4hA_2 (removed from memory)
31/07/2005 03:48:47|LHC@home|Restarting result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 using sixtrack version 4.67
31/07/2005 04:36:03|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:36:03|LHC@home|Requesting 1 seconds of work, returning 0 results
31/07/2005 04:36:04|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:36:04|LHC@home|No schedulers responded
31/07/2005 04:36:05|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 04:37:05|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:37:05|LHC@home|Requesting 65 seconds of work, returning 0 results
31/07/2005 04:37:06|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:37:06|LHC@home|No schedulers responded
31/07/2005 04:37:07|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 04:38:06|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:38:06|LHC@home|Requesting 128 seconds of work, returning 0 results
31/07/2005 04:38:07|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:38:07|LHC@home|No schedulers responded
31/07/2005 04:38:08|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 04:39:07|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:39:07|LHC@home|Requesting 192 seconds of work, returning 0 results
31/07/2005 04:39:08|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:39:08|LHC@home|No schedulers responded
31/07/2005 04:39:09|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 04:40:08|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:40:08|LHC@home|Requesting 255 seconds of work, returning 0 results
31/07/2005 04:40:09|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:40:09|LHC@home|No schedulers responded
31/07/2005 04:40:10|LHC@home|Deferring communication with project for 2 minutes and 12 seconds
31/07/2005 04:42:23|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:42:23|LHC@home|Requesting 395 seconds of work, returning 0 results
31/07/2005 04:42:24|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:42:24|LHC@home|No schedulers responded
31/07/2005 04:42:26|LHC@home|Deferring communication with project for 3 minutes and 18 seconds
31/07/2005 04:45:45|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:45:45|LHC@home|Requesting 604 seconds of work, returning 0 results
31/07/2005 04:45:46|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:45:46|LHC@home|No schedulers responded
31/07/2005 04:45:47|LHC@home|Deferring communication with project for 8 minutes and 17 seconds
31/07/2005 04:54:05|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 04:54:05|LHC@home|Requesting 1121 seconds of work, returning 0 results
31/07/2005 04:54:06|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 04:54:06|LHC@home|No schedulers responded
31/07/2005 04:54:08|LHC@home|Deferring communication with project for 29 minutes and 26 seconds
31/07/2005 05:23:35|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 05:23:35|LHC@home|Requesting 2945 seconds of work, returning 0 results
31/07/2005 05:23:36|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 05:23:36|LHC@home|No schedulers responded
31/07/2005 05:23:37|LHC@home|Deferring communication with project for 1 hours, 21 minutes, and 53 seconds
31/07/2005 06:23:37|LHC@home|Deferring communication with project for 21 minutes and 52 seconds
31/07/2005 06:45:31|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 06:45:31|LHC@home|Requesting 8044 seconds of work, returning 0 results
31/07/2005 06:45:32|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 06:45:32|LHC@home|No schedulers responded
31/07/2005 06:45:33|LHC@home|Deferring communication with project for 40 minutes and 3 seconds
31/07/2005 06:55:08||request_reschedule_cpus: process exited
31/07/2005 06:55:08|LHC@home|Computation for result wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4 finished
31/07/2005 06:55:08|Einstein@Home|Resuming result w1_0876.5__0876.9_0.1_T00_S4hA_2 using einstein version 4.79
31/07/2005 06:55:10|LHC@home|Started upload of wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4_0
31/07/2005 06:55:14|LHC@home|Finished upload of wjun1C_v6s4hvnom_mqx__19__64.297_59.307__4_6__6__5_1_sixvf_boinc24920_4_0
31/07/2005 06:55:14|LHC@home|Throughput 12177 bytes/sec
31/07/2005 07:10:03||request_reschedule_cpus: project op
31/07/2005 07:10:03|Einstein@Home|Sending scheduler request to http://einstein.phys.uwm.edu/EinsteinAtHome_cgi/cgi
31/07/2005 07:10:03|Einstein@Home|Requesting 0 seconds of work, returning 0 results
31/07/2005 07:10:06|Einstein@Home|Scheduler request to http://einstein.phys.uwm.edu/EinsteinAtHome_cgi/cgi succeeded
31/07/2005 07:13:00||request_reschedule_cpus: project op
31/07/2005 07:13:01|Einstein@Home|Sending scheduler request to http://einstein.phys.uwm.edu/EinsteinAtHome_cgi/cgi
31/07/2005 07:13:01|Einstein@Home|Requesting 0 seconds of work, returning 0 results
31/07/2005 07:13:03|Einstein@Home|Scheduler request to http://einstein.phys.uwm.edu/EinsteinAtHome_cgi/cgi succeeded
31/07/2005 07:13:03|Einstein@Home|General preferences have been updated
31/07/2005 07:13:03||General prefs: from Einstein@Home (last modified 2005-07-31 07:10:28)
31/07/2005 07:13:03||General prefs: using separate prefs for work
31/07/2005 07:25:36|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:25:36|LHC@home|Requesting 8640 seconds of work, returning 1 results
31/07/2005 07:25:37|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:25:37|LHC@home|No schedulers responded
31/07/2005 07:25:38|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 07:26:38|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:26:38|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:26:39|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:26:39|LHC@home|No schedulers responded
31/07/2005 07:26:40|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 07:27:39|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:27:39|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:27:40|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:27:40|LHC@home|No schedulers responded
31/07/2005 07:27:41|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 07:28:41|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:28:41|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:28:42|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:28:42|LHC@home|No schedulers responded
31/07/2005 07:28:43|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 07:29:42|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:29:42|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:29:43|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:29:43|LHC@home|No schedulers responded
31/07/2005 07:29:45|LHC@home|Deferring communication with project for 1 minutes and 26 seconds
31/07/2005 07:31:12|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:31:12|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:31:13|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:31:13|LHC@home|No schedulers responded
31/07/2005 07:31:14|LHC@home|Deferring communication with project for 5 minutes and 7 seconds
31/07/2005 07:36:22|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:36:22|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:36:23|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:36:23|LHC@home|No schedulers responded
31/07/2005 07:36:24|LHC@home|Deferring communication with project for 8 minutes and 53 seconds
31/07/2005 07:45:17|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:45:17|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:45:18|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:45:18|LHC@home|No schedulers responded
31/07/2005 07:45:19|LHC@home|Deferring communication with project for 8 minutes and 3 seconds
31/07/2005 07:53:23|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 07:53:23|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 07:53:24|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 07:53:24|LHC@home|No schedulers responded
31/07/2005 07:53:25|LHC@home|Deferring communication with project for 1 hours, 22 minutes, and 4 seconds
31/07/2005 08:53:25|LHC@home|Deferring communication with project for 22 minutes and 4 seconds
31/07/2005 09:15:30|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 09:15:30|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 09:15:31|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 09:15:31|LHC@home|No schedulers responded
31/07/2005 09:15:32|LHC@home|Deferring communication with project for 2 hours, 19 minutes, and 25 seconds
31/07/2005 10:15:33|LHC@home|Deferring communication with project for 1 hours, 19 minutes, and 24 seconds
31/07/2005 11:15:34|LHC@home|Deferring communication with project for 19 minutes and 23 seconds
31/07/2005 11:34:57|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:34:57|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:34:58|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:34:58|LHC@home|No schedulers responded
31/07/2005 11:34:59|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 11:35:59|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:35:59|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:36:00|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:36:00|LHC@home|No schedulers responded
31/07/2005 11:36:01|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 11:37:00|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:37:00|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:37:01|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:37:01|LHC@home|No schedulers responded
31/07/2005 11:37:02|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 11:38:01|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:38:01|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:38:02|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:38:02|LHC@home|No schedulers responded
31/07/2005 11:38:03|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 11:39:03|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:39:03|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:39:04|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:39:04|LHC@home|No schedulers responded
31/07/2005 11:39:05|LHC@home|Deferring communication with project for 1 minutes and 28 seconds
31/07/2005 11:40:33|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:40:33|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:40:34|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:40:34|LHC@home|No schedulers responded
31/07/2005 11:40:35|LHC@home|Deferring communication with project for 2 minutes and 56 seconds
31/07/2005 11:43:32|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:43:32|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:43:33|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:43:33|LHC@home|No schedulers responded
31/07/2005 11:43:34|LHC@home|Deferring communication with project for 3 minutes and 16 seconds
31/07/2005 11:46:51|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 11:46:51|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 11:46:52|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 11:46:52|LHC@home|No schedulers responded
31/07/2005 11:46:54|LHC@home|Deferring communication with project for 46 minutes and 6 seconds
31/07/2005 12:33:01|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 12:33:01|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 12:33:02|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 12:33:02|LHC@home|No schedulers responded
31/07/2005 12:33:03|LHC@home|Deferring communication with project for 1 hours, 27 minutes, and 11 seconds
31/07/2005 13:33:03|LHC@home|Deferring communication with project for 27 minutes and 10 seconds
31/07/2005 14:00:14|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 14:00:14|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 14:00:15|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 14:00:15|LHC@home|No schedulers responded
31/07/2005 14:00:16|LHC@home|Deferring communication with project for 3 hours, 34 minutes, and 24 seconds
31/07/2005 15:00:17|LHC@home|Deferring communication with project for 2 hours, 34 minutes, and 23 seconds
31/07/2005 16:00:18|LHC@home|Deferring communication with project for 1 hours, 34 minutes, and 22 seconds
31/07/2005 17:00:18|LHC@home|Deferring communication with project for 34 minutes and 22 seconds
31/07/2005 17:34:41|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:34:41|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:34:42|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:34:42|LHC@home|No schedulers responded
31/07/2005 17:34:43|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:35:42|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:35:42|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:35:43|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:35:43|LHC@home|No schedulers responded
31/07/2005 17:35:44|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:36:43|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:36:43|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:36:44|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:36:44|LHC@home|No schedulers responded
31/07/2005 17:36:45|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:37:44|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:37:44|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:37:46|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:37:46|LHC@home|No schedulers responded
31/07/2005 17:37:47|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:38:46|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:38:46|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:38:47|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:38:47|LHC@home|No schedulers responded
31/07/2005 17:38:48|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:39:47|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:39:47|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:39:48|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:39:48|LHC@home|No schedulers responded
31/07/2005 17:39:49|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:40:48|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:40:48|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:40:49|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:40:49|LHC@home|No schedulers responded
31/07/2005 17:40:50|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:41:49|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:41:49|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:41:51|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:41:51|LHC@home|No schedulers responded
31/07/2005 17:41:52|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:42:51|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:42:51|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:42:52|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:42:52|LHC@home|No schedulers responded
31/07/2005 17:42:53|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:43:52|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:43:52|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:43:53|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:43:53|LHC@home|No schedulers responded
31/07/2005 17:43:54|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:44:53|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:44:53|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:44:54|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:44:54|LHC@home|No schedulers responded
31/07/2005 17:44:55|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:45:54|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:45:54|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:45:56|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:45:56|LHC@home|No schedulers responded
31/07/2005 17:45:57|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:46:56|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:46:56|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:46:57|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:46:57|LHC@home|No schedulers responded
31/07/2005 17:46:58|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:47:57|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:47:57|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:47:58|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:47:58|LHC@home|No schedulers responded
31/07/2005 17:47:59|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:48:58|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:48:58|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:48:59|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:48:59|LHC@home|No schedulers responded
31/07/2005 17:49:00|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:49:59|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:49:59|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:50:01|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:50:01|LHC@home|No schedulers responded
31/07/2005 17:50:02|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:51:01|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:51:01|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:51:02|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:51:02|LHC@home|No schedulers responded
31/07/2005 17:51:03|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:52:02|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:52:02|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:52:03|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:52:03|LHC@home|No schedulers responded
31/07/2005 17:52:04|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:53:03|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:53:03|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:53:04|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:53:04|LHC@home|No schedulers responded
31/07/2005 17:53:05|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:54:04|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:54:04|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:54:06|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:54:06|LHC@home|No schedulers responded
31/07/2005 17:54:07|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:55:06|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:55:06|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:55:07|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:55:07|LHC@home|No schedulers responded
31/07/2005 17:55:08|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:56:07|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:56:07|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:56:08|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:56:08|LHC@home|No schedulers responded
31/07/2005 17:56:09|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:57:08|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:57:08|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:57:09|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:57:09|LHC@home|No schedulers responded
31/07/2005 17:57:10|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:58:09|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:58:09|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:58:11|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:58:11|LHC@home|No schedulers responded
31/07/2005 17:58:12|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 17:59:11|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 17:59:11|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 17:59:12|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 17:59:12|LHC@home|No schedulers responded
31/07/2005 17:59:13|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:00:12|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:00:12|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:00:13|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:00:13|LHC@home|No schedulers responded
31/07/2005 18:00:14|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:01:13|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:01:13|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:01:14|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:01:14|LHC@home|No schedulers responded
31/07/2005 18:01:15|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:02:14|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:02:14|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:02:16|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:02:16|LHC@home|No schedulers responded
31/07/2005 18:02:17|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:03:16|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:03:16|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:03:17|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:03:17|LHC@home|No schedulers responded
31/07/2005 18:03:18|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:04:17|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:04:17|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:04:18|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:04:18|LHC@home|No schedulers responded
31/07/2005 18:04:19|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:05:18|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:05:18|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:05:19|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:05:19|LHC@home|No schedulers responded
31/07/2005 18:05:20|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:06:19|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:06:19|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:06:20|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:06:20|LHC@home|No schedulers responded
31/07/2005 18:06:22|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:07:21|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:07:21|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:07:22|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:07:22|LHC@home|No schedulers responded
31/07/2005 18:07:23|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:08:22|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:08:22|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:08:23|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:08:23|LHC@home|No schedulers responded
31/07/2005 18:08:24|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:09:23|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:09:23|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:09:24|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:09:24|LHC@home|No schedulers responded
31/07/2005 18:09:25|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:10:24|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:10:24|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:10:25|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:10:25|LHC@home|No schedulers responded
31/07/2005 18:10:27|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:11:26|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:11:26|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:11:27|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:11:27|LHC@home|No schedulers responded
31/07/2005 18:11:28|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:12:27|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:12:27|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:12:28|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:12:28|LHC@home|No schedulers responded
31/07/2005 18:12:29|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:13:28|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:13:28|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:13:29|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:13:29|LHC@home|No schedulers responded
31/07/2005 18:13:30|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:14:29|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:14:29|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:14:30|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:14:30|LHC@home|No schedulers responded
31/07/2005 18:14:32|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:15:31|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:15:31|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:15:32|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:15:32|LHC@home|No schedulers responded
31/07/2005 18:15:33|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:16:32|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:16:32|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:16:33|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:16:33|LHC@home|No schedulers responded
31/07/2005 18:16:34|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:17:33|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:17:33|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:17:34|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:17:34|LHC@home|No schedulers responded
31/07/2005 18:17:35|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:18:34|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:18:34|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:18:35|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:18:35|LHC@home|No schedulers responded
31/07/2005 18:18:36|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:19:36|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:19:36|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:19:37|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:19:37|LHC@home|No schedulers responded
31/07/2005 18:19:38|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:20:37|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:20:37|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:20:38|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:20:38|LHC@home|No schedulers responded
31/07/2005 18:20:39|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:21:38|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:21:38|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:21:39|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:21:39|LHC@home|No schedulers responded
31/07/2005 18:21:40|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:22:39|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:22:39|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:22:40|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:22:40|LHC@home|No schedulers responded
31/07/2005 18:22:41|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:23:41|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:23:41|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:23:42|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:23:42|LHC@home|No schedulers responded
31/07/2005 18:23:43|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:24:42|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:24:42|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:24:43|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:24:43|LHC@home|No schedulers responded
31/07/2005 18:24:44|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:25:43|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:25:43|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:25:44|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:25:44|LHC@home|No schedulers responded
31/07/2005 18:25:45|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:26:44|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:26:44|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:26:45|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:26:45|LHC@home|No schedulers responded
31/07/2005 18:26:46|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:27:46|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:27:46|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:27:48|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:27:48|LHC@home|No schedulers responded
31/07/2005 18:27:49|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:28:48|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:28:48|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:28:49|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:28:49|LHC@home|No schedulers responded
31/07/2005 18:28:50|LHC@home|Deferring communication with project for 59 seconds
31/07/2005 18:29:49|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:29:49|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:29:51|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:29:51|LHC@home|No schedulers responded
31/07/2005 18:29:52|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:30:51|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:30:51|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:30:52|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:30:52|LHC@home|No schedulers responded
31/07/2005 18:30:53|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:31:52|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:31:52|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:31:53|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:31:53|LHC@home|No schedulers responded
31/07/2005 18:31:55|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:32:54|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:32:54|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:32:55|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:32:55|LHC@home|No schedulers responded
31/07/2005 18:32:56|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:33:56|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:33:56|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:33:58|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:33:58|LHC@home|No schedulers responded
31/07/2005 18:33:59|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:34:58|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:34:58|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:34:59|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:34:59|LHC@home|No schedulers responded
31/07/2005 18:35:00|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:36:00|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:36:00|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:36:02|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:36:02|LHC@home|No schedulers responded
31/07/2005 18:36:03|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:37:02|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:37:02|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:37:04|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:37:04|LHC@home|No schedulers responded
31/07/2005 18:37:05|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:38:04|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:38:04|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:38:05|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:38:05|LHC@home|No schedulers responded
31/07/2005 18:38:07|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:39:06|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:39:06|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:39:07|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:39:07|LHC@home|No schedulers responded
31/07/2005 18:39:08|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:40:07|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:40:07|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:40:31|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:40:31|LHC@home|No schedulers responded
31/07/2005 18:40:32|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:41:31|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:41:31|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:41:55|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:41:55|LHC@home|No schedulers responded
31/07/2005 18:41:56|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:42:55|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:42:55|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:43:18|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:43:18|LHC@home|No schedulers responded
31/07/2005 18:43:19|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:44:18|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:44:18|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:44:42|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:44:42|LHC@home|No schedulers responded
31/07/2005 18:44:43|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:45:42|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:45:42|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:46:06|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:46:06|LHC@home|No schedulers responded
31/07/2005 18:46:07|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:47:06|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:47:06|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:47:30|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:47:30|LHC@home|No schedulers responded
31/07/2005 18:47:31|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:48:30|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:48:30|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:48:54|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:48:54|LHC@home|No schedulers responded
31/07/2005 18:48:55|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:49:54|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:49:54|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:50:18|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:50:18|LHC@home|No schedulers responded
31/07/2005 18:50:19|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:51:19|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:51:19|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:51:43|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:51:43|LHC@home|No schedulers responded
31/07/2005 18:51:44|LHC@home|Deferring communication with project for 58 seconds
31/07/2005 18:52:43|LHC@home|Sending scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi
31/07/2005 18:52:43|LHC@home|Requesting 0 seconds of work, returning 1 results
31/07/2005 18:53:07|LHC@home|Scheduler request to http://lhcathome-sched1.cern.ch/scheduler/cgi failed
31/07/2005 18:53:07|LHC@home|No schedulers responded
31/07/2005 18:53:08|LHC@home|Deferring communication with project for 58 seconds
[/pre]
~~gravywavy