Fermi LAT Gamma-ray pulsar search #4 "FGRP4"

astro-marwil
astro-marwil
Joined: 28 May 05
Posts: 532
Credit: 648876543
RAC: 1112627

Expected crunching time much

Expected crunching time much too high !

Hallo!
I do crunch fgrp4 tasks within about 6:15h constantly but the expected crunching time set by the program do vary around 41h. That´s not very problematic, as I have always a buffer of tasks for about 8h downloaded only, but I like to have a buffer for 3 days. This isn´t possible with this unrealistic long calculated expected crunching time.
I hope it became clear, what I ment, and it can become cured.

Kind regards and happy crunching
Martin

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1364
Credit: 3562358667
RAC: 0

They're about 10 vs 60h on my

They're about 10 vs 60h on my box (down from 75 initially); but the BRP5 tasks pulling DCF the other way; I don't expect it to get much better. Boinc's use of a single DCF value for the entire project instead of a per app one was why I abandoned FGRP3 early on; and will probably pull the plug on FGRP4 as soon as the next GW search comes on line. Mostly through luck, the GW and BRP projects have had reasonably similar DCFs on my boxes and do play nicely with each other.

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1364
Credit: 3562358667
RAC: 0

OK, something's changed my

OK, something's changed my BRP5 and FRGP4 tasks are now both showing approximately correct ETAs in BOINC Manager; did E@H tweak the base task duration for FRPG4 to be closer to the baseline from the GW tasks?

Maximilian Mieth
Maximilian Mieth
Joined: 4 Oct 12
Posts: 130
Credit: 10298249
RAC: 4354

RE: OK, something's changed

Quote:
OK, something's changed my BRP5 and FRGP4 tasks are now both showing approximately correct ETAs in BOINC Manager; did E@H tweak the base task duration for FRPG4 to be closer to the baseline from the GW tasks?


Others report similar observations.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4312
Credit: 250570845
RAC: 34530

RE: did E@H tweak the base

Quote:
did E@H tweak the base task duration for FRPG4 to be closer to the baseline from the GW tasks?

Yes.

BM

BM

Phil
Phil
Joined: 8 Jun 14
Posts: 583
Credit: 228552404
RAC: 5426

Ah, that explains a

Ah, that explains a lot.

Thanks for the info Bernd.

Phil

I thought I was wrong once, but I was mistaken.

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1364
Credit: 3562358667
RAC: 0

Thank you.

Thank you.

Stranger7777
Stranger7777
Joined: 17 Mar 05
Posts: 436
Credit: 429926359
RAC: 77889

It looks like FGRP4 project

It looks like FGRP4 project has got additional work from Fermi and now it have more time to live. This is great! But it uses way too much memory and is unsuitable for almost all 32-bit machines I crunch on. Thanks developers you have GW applications that use 10 times less memory and are very comfortable to me. That is why I have to deprecate the use of FGRP4 tasks for now on my hosts. I understand that memory requirements can't be lowered for CUDA and OpenCL tasks due to the way programs communicate with GPU effectively. So I'm ready to return to this project in 4 cases:
1) FGRP4 will reduce memory requirements for about 50% at least;
2) There will be no more CPU work for GW and consequent searches;
3) GW searches will use the same amount of memory as FGRP does but will be less productive;
4) I will install additional memory on my hosts or even move them to 64-bit (this is not suitable for me yet, that's why this will be the last thing I will do).
Are there any plans on making this project a little bit more suitable for oldsckoolers like me?

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5872
Credit: 117681015931
RAC: 35112318

RE: 1) FGRP4 will reduce

Quote:
1) FGRP4 will reduce memory requirements for about 50% at least;


I very much doubt that the app is deliberately wasteful of memory. I'm sure there would be a very good reason for it being the way it is.

Quote:
2) There will be no more CPU work for GW and consequent searches;


I imagine GW work will explode when advanced LIGO starts delivering :-).

Quote:
4) I will install additional memory on my hosts or even move them to 64-bit (this is not suitable for me yet, that's why this will be the last thing I will do).


It might not be your "last thing" alternative if we could somehow change the "not suitable for me yet" bit. Maybe you'd like to explain why it's unsuitable? I'd certainly like to help with your problem if I can.

There are probably lots of 512MB and 1GB DDR2 sticks lying around in people's spare parts boxes. I know I've got 6x512MB and probably a few 1GB (all DDR2) that I'm quite unlikely to use again. I've even got plain DDR. There's probably lots of these on ebay going for quite low price because for most people they would be 'surplus to requirements'. Do your machines with low amounts of RAM have spare slots? Do they take DDR2 RAM?

The other thing you could do to economise on memory is not run a desktop environment. You could load a 64bit Linux but just use a console login and launch BOINC from that. You could manage such machines remotely using BOINC Manager on a single machine that is running a DE, eg a Windows machine if you wished.

Cheers,
Gary.

Stranger7777
Stranger7777
Joined: 17 Mar 05
Posts: 436
Credit: 429926359
RAC: 77889

RE: I'm sure there would be

Quote:
I'm sure there would be a very good reason for it being the way it is.


I think too, so I have to dismiss from that for a while.

Quote:
Quote:
2) There will be no more CPU work for GW and consequent searches;

I imagine GW work will explode when advanced LIGO starts delivering :-).


I'm currently concentrating on finishing FU#1. But server doesn't want to feed me enough, only minimum that is necessary to supply CPU with tasks. Though I know why, so it is not a complaint :)

Quote:
Maybe you'd like to explain why it's unsuitable? I'd certainly like to help with your problem if I can.


DDR2 machines are quite modern for us and they are working in offices. But DDR2 modules are not cheap for me due to cost of a $. Moreover these machines have only one or 2 slots. Almost any of them have max 512Mb frequently used also for internal video. It was and is enough to work in Win XP and in my warehouse and trading program. When we talking about 1-core Celerons then it is really enough even for GW. But speaking of Core2 Quad or even of Dual-Core Celerons and Pentiums - this amount already is not enough to work on all cores. So I have to set the maximum of usable cores to limit the amount of memory for BOINC (limiting BOINC itself with memory settings doesn't work well for me, because it measures only itself and WU's, but doesn't look after users programs).
I don't want to manage all these machines, but want to run them flawlessly without such limitations. But this way I should install at least 512 Mb for user, 512 Mb for internal video and 512 Mb for each core or GPU. So it must be not less then 1 Gb. 2 Gb is enough right now for 2 core machines for almost any current tasks user can wish. And 3-4 Gb (of course 4 to run dual-channel, 4Gb modules are very-very expensive and don't want to run in older chipsets like i945 etc.) is enough to run 32-bit Windows on 4-cores machines. Don't persuade me to install a 64-bit OS. There are several reasons for that - absence of NTVDM and using still less than 4 Gb on 4 Gb machines (due to chipset or internal video limitation). Having only 1 or 2 slots for memory I can not afford 512 Mb modules for upgrade. But 1,2,4 Gb modules are very expensive, so neither customers nor me don't want to pay for that.
For other machines with DDR1 I found lots of 128-256-512 modules that I tend to use until they die :) (that is why I'm asking devs to produce CPU work for as long as it possible). But they also have 2 (max 3) slots, so for one core machine (Celeron, P4) that is working only on GW an FGRP4 768 Mb is enough when it is not used by users. On the other way I have to install 2x1 Gb DDR modules to make the machine comfortable for user (though they are already not because of toooo sloooow java and flash technologies used almost on any site).

Speaking about fresh rigs like Pentium G, Core i3, Core i5 and Celerons, they already have 4 Gb of RAM and 32-bit OS as I told here. So BOINC have to huddle with other applications, Intel video that uses much of this memory, and 3-4 instances of FGRP4 and 1-2 instances of BRP OpenCL tasks that use more than 400 MBs too. I try to equip these rigs with SSD to smooth the effect of swapping. But even reducing memory requirement to 300 Mb would solve this problem on almost any 32-bit machine. Again, I have a reason to use 32-bit Windows. I can not run my application in 64-bit environment and will not build VMs for this. Users just will not accept such complication. They love my program for its simplicity.
So, these days GW is the app that I can run on all of these machines without any intervention and complaints from users all over my and nearest (~300 km) cities. Most of my customers don't even have any internet connection - they simply don't need it. But almost half of them who do, they have 3G-4G USB-modems that eats with its soft about 200 Mb of RAM and almost all CPU power of a Northwood Celeron and still works like 2.5G connection (long and lost pings, long connection time, poor data transfer) due to quality of service provided by operators and huge distances between base stations, so I have to lower the priority of modem soft to idle to make machine usable, and have to refuse myself to use remote administration like with TeamViewer. Though I found myself an old RAdmin soft that works fine enough sometimes, but to connect those hosts I have to build VPN with Hamachi that now with the 2nd version uses 30 MB of memory too.
This is very expensive for me to feed all these rigs with suitable memory modules. That's why I'm asking... :) It would be good for me if it happened.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.