Suspend Project, preferably GPU's, Certain Hours During Weekdays

Robby
Robby
Joined: 18 Jan 05
Posts: 33
Credit: 395123303
RAC: 643
Topic 213525

Howdy Crunchers,

APS electricity rates have almost doubled!  Is there a way to SUSPEND operation say from 3pm to 8pm, peak hours, during weekdays Monday through Friday, preferably just GPU's.  Daily schedule only allows you to compute during certain hours but no way to suspend.  The only solution I can think of is to run a cron job from 3pm to 8pm and have all crunching suspend while some task runs that suspends Boinc.  Would prefer to just suspend GPU's from 3pm to 8pm.  Running Mint 18.3 linux on 2 Ryzen 8 core 16 thread machines with 2 1050ti GPU's each, plus a Vishera 8350 8 core machine with 2 older 620, possibly will upgrade to 750ti, GPU's, also running Mint.  THX

archae86
archae86
Joined: 6 Dec 05
Posts: 3146
Credit: 7086294931
RAC: 1338292

Robby_10 wrote:Daily schedule

Robby_10 wrote:
Daily schedule only allows you to compute during certain hours but no way to suspend.

Suspend is just the complement of compute, regarding scheduling, yes?

Granted you don't get the day of week dependence, nor the GPU vs. CPU work dependence from this, but you do get to divide the day into two contiguous segments, one with compute enabled, one suspended.

 

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5851
Credit: 110685784178
RAC: 32932694

Robby_10 wrote:... Is there a

Robby_10 wrote:
... Is there a way to SUSPEND operation say from 3pm to 8pm, peak hours, during weekdays Monday through Friday, preferably just GPU's.

Seeing as you are running Linux, you can easily write small scripts to get boinccmd to independently control CPU and GPU.  Look in the documentation for the separate control options --set_run_mode and --set_gpu_mode.  These options take a single argument, one of { auto | always | never } with a further optional [duration].  If you give a duration, the mode change gets reverted at the end of that duration.  So you could set up a cron job to launch the script at 3pm (using 'never' to suspend crunching) and with the appropriate duration (18000) included, it would resume crunching at 8pm after 5 hrs had elapsed.

I have a little bit of experience at this sort of thing but not using cron.  Maintaining a cron script on a very large number of individual machines is not a very appealing prospect.  Running a script on a single machine and having that script talk to all machines on the LAN over ssh connections is much more efficient for me.  I do it for temperature control during heat waves.  I don't use aircon to keep machines cool - just forced ventilation.  If the ambient in the machine room gets above 38C, a script gets launched on a server machine which suspends all crunching.  The forced ventilation works quite quickly to bring the temperature down to a more reasonable level.

I could be wrong but I suspect that you may see a better power saving by stopping CPU crunching rather than GPU crunching.  Your 1050Tis would be quite productive (I'm guessing from what others say - I don't own any) and quite miserly on power used.  In particular, your 8 core 8350 would use lots of power when crunching CPU tasks on all available cores.  Unfortunately 750Tis are not very productive with current GPU tasks.  I suspect that you would find that a single 1050Ti would use much less power and produce more output than 2 750Tis.

Do you happen to have access to a kill-a-watt style power meter?  If you do, you should try using it to measure the power saving you could achieve.  Start off by measuring the power consumption at the wall with everything running normally.  Then select and suspend all onboard 'ready to start' tasks (both CPU and GPU) so that new ones can't start when you suspend running tasks.  If you then suspend the running GPU tasks, you will be able to measure how much you are saving by stopping GPU tasks.

When you have that new figure, suspend the running CPU tasks so that nothing is now crunching.  You will see a further drop in power consumption which may give you enough information.  You could then resume just the GPU tasks that were running previously and measure that power increase before resuming everything and going back to full normal crunching.  From all the figures collected, you should be able to work out your best option for saving power without crippling your completed task throughput.

Remember, there is a lot more science being done in a FGRP GPU task than a FGRP CPU task.  You can tell this from the different credit awards.

 

Quote:
... while some task runs that suspends Boinc.

You can't suspend BOINC without interfering with all crunching.  You suspend a particular science application but BOINC must continue to run to look after apps that aren't suspended and to manage things when a suspended app gets resumed.  BOINC itself consumes very little power anyway.

 

Cheers,
Gary.

Robby
Robby
Joined: 18 Jan 05
Posts: 33
Credit: 395123303
RAC: 643

Got it working.  Using kalarm

Got it working.  Using kalarm in Kde or "alarm clock" in gnome, set a command alarm to start say psensor at 3pm and have Boinc suspend the GPU's through the exclusive applications option.  Then send out another command alarm at 8pm with pkill psensor to stop psensor which will restart the GPU's.  Would have preferred to stop the CPU's but doing so also stops the GPU's which need the CPU's to feed them. Saving about 60 watts per 1050ti for the 5 hours.  Thanks for the inputs.  May change over to cron jobs if everything keeps working okay.

AgentB
AgentB
Joined: 17 Mar 12
Posts: 915
Credit: 513211304
RAC: 0

I run one cruncher from 23:00

I run one cruncher from 23:00 until 7 am, when it shuts down and restarts at 23:00 later that day.  The power savings are significant for me during that time.    rtcwake is the magic command.

My root crontab entry is...

# at 7 am shutdown until 11 pm

0 7 * * * /usr/sbin/rtcwake -m off --date 23:00:00

I have some other scripts which shutdown first the GPU crunching, then CPU crunching, then power off  for a while when room temperature gets to certain levels.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.