Discussion Thread for the Continuous GW Search known as O2MD1 (now O2MDF - GPUs only)

Eskomorko
Eskomorko
Joined: 15 Jan 09
Posts: 39
Credit: 840632717
RAC: 566889

Has anyone tried those 1.09

Has anyone tried those 1.09 GPU apps yet? They seem to take a while to complete. I was parallel-running those and it took 9hrs per wu to complete with RTX2070.

robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454484533
RAC: 8476

I am seeing about 9 hours per

I am seeing about 9 hours per WU (Gravitational Wave search O2 Multi-Directional v1.01 () x86_64-pc-linux-gnu) on a Linux PC (AuthenticAMD AMD Ryzen 7 1700 Eight-Core Process () x86_64-pc-linux-gnuor [Family 23 Model 1 Stepping 1] (16 processors) . 

What is strange to me is that I have 4 pending waiting for validation. They have been in this state for about 3 days.  No credits granted.  

Jim1348
Jim1348
Joined: 19 Jan 06
Posts: 463
Credit: 257957147
RAC: 0

Eskomorko wrote:Has anyone

Eskomorko wrote:
Has anyone tried those 1.09 GPU apps yet? They seem to take a while to complete. I was parallel-running those and it took 9hrs per wu to complete with RTX2070.

https://einsteinathome.org/content/discussion-thread-continuous-gw-search-known-o2md1?page=2#comment-173647

Holmis
Joined: 4 Jan 05
Posts: 1118
Credit: 1055935564
RAC: 0

robl wrote:What is strange to

robl wrote:
What is strange to me is that I have 4 pending waiting for validation. They have been in this state for about 3 days.  No credits granted.  

Nothing strange at all if you read Bernd's announcement in Tech News: https://einsteinathome.org/content/gravitational-wave-search-o2-multi-directional-o2md1

Quote:
- validation will be started on Monday on the results that have been reported until then
robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454484533
RAC: 8476

Holmis wrote:robl wrote:What

Holmis wrote:
robl wrote:
What is strange to me is that I have 4 pending waiting for validation. They have been in this state for about 3 days.  No credits granted.  

Nothing strange at all if you read Bernd's announcement in Tech News: https://einsteinathome.org/content/gravitational-wave-search-o2-multi-directional-o2md1

Quote:
- validation will be started on Monday on the results that have been reported until then

[/quote

Holmis, thanks.  I remember reading that post but did not retain the info within.  

Richie
Richie
Joined: 7 Mar 14
Posts: 656
Credit: 1702989778
RAC: 0

I'm sure now that O2MD1 v1.01

I'm sure now that O2MD1 v1.01 run times depend very much on the frequency that is included at the end part of a task name (xx.xxHz). The range has been at least from 21.xx to 77.xx Hz. Lower freq tasks run faster. If 21.xx Hz represents basic level of 100 % run time then 77.xx Hz tasks have run about 500 % as long. So there's a clear freq-runtime-curve, perhaps similar to what I remember user Archae86 has shown in the past with another app.

I wonder if there's going to be new batches of work now and then... so that the first tasks of a new O2MD1 batch would start again from low freq and then the freq band would shift up as more tasks were sent out.

Zalster
Zalster
Joined: 26 Nov 13
Posts: 3117
Credit: 4050672230
RAC: 0

Haven't seen any O2MD1 yet,

Haven't seen any O2MD1 yet, all I keep getting are the O2AS20-500.  

Zalster
Zalster
Joined: 26 Nov 13
Posts: 3117
Credit: 4050672230
RAC: 0

Richie wrote:Richie

Richie wrote:
Richie wrote:
O2MD1 v1.01 cpu tasks seem to run about 20 % faster than O2AS v1.01 cpu tasks.

I'm pulling that statement back. Can't make that kind of comparison. Most of the tasks that I've been watching so far have completed in about 9-12h for example. But I noticed now  there are some black sheeps that manage to complete the race in under 2 hours on that same host X. Perhaps the frequency bands of these tasks do have a hefty connection with the run times, again.

edit: relative run time examples from host X

57 Hz ... 33k
41 Hz ... 26k
21 Hz ... 7k


Richie, can you post a link to your results so I can see them?  Thanks

 

Z

Richie
Richie
Joined: 7 Mar 14
Posts: 656
Credit: 1702989778
RAC: 0

Zalster wrote:Richie

Zalster wrote:
Richie wrote:

edit: relative run time examples from host X

57 Hz ... 33k
41 Hz ... 26k
21 Hz ... 7k


Richie, can you post a link to your results so I can see them?  Thanks

 Z

Hi! "X" is this host: https://einsteinathome.org/host/12761897
It's been crunching 3x O2MD1 v1.01 cpu + 1x O2AS v1.09 gpu tasks concurrently. Those run times that I mentioned can be found now at the 'pending'. Ps. Two tasks got 'error while computing' yesterday. It happened after I decided to reboot the host because there were some Windows updates pending for a reboot. After I had started Boinc again and allowed crunching... that instant load of all 4 tasks at once was somekind of a shock for the system. Screen was full of artifacts, really a complete mess for a couple of seconds. Then the first task errored out. Then the second one... until I managed to hit stop. I've seen that somewhere in the past, but maybe with another host. I knew that host wouldn't recover from that and would've just crashed all the other tasks too. But then I made another reboot to make sure the condition is reset. I decided to start only 2 tasks at first and rest of the tasks a few seconds later... to decrease the system stress at the moment when crunching starts. It worked well and all is good again. I haven't seen that earlier with tasks from another apps or another kind of mixed work that this host was doing lately... and I didn't try to reproduce that problem now. In that way it surprised me a bit. Do these tasks stress the system in a different way, I don't know.

edit: To be clear, it was AS20 gpu tasks that were crashing, not these O2MD1 tasks. I suspect that problem had just something to do with AMD gpu driver or system and didn't have anything particular to do with these applications.

I've got total 171 O2MD1 tasks pending now. 6 hosts have been running same sort of mixed set (O2MD1 cpu tasks + 1x AS20 gpu) and two hosts (1 windows, 1 linux) have been running O2MD1 tasks only.

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4704
Credit: 17548965530
RAC: 6434659

Gary Roberts wrote:Keith

Gary Roberts wrote:
Keith Myers wrote:
I wish I had kept the entire scheduler connect log.

That shouldn't be necessary as the last contact is always available and you would expect to see some sort of continuing problem with the O2MD1 app or plan class being mentioned each time.

I've just looked through all the most recent scheduler logs for all your hosts and can't find any of them referring in any way, good, bad or indifferent, to O2MD1.  Which particular host ID is the one you wish to use for the new search?  If you let me know which one it is, I can look at the log without bothering you to post the whole deal, or even a link to it.

Gary, I still have not seen any log resembling what I first saw, but I do a similar log from Zalster currently.  He is not getting any O2MD1 cpu tasks either.

https://einsteinathome.org/host/12789230/log

 

My log had even more stuff in it about locality scheduling and other scheduling restrictions and what apps were valid etc.

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.