Automating changes to task multiples

A new and improved version, taskXDF and related files, is up on the Dropbox page,

https://www.dropbox.com/sh/kiftk65mg59ezm6/AACeHeeKYvvrj65VawsRM107a?dl=0

In addition to improved look-ahead functions for matching VRAM usage to task multiplicities, a timer script can now be used for running the script on timed intervals; no more need for systemd.

Given the slow release of tasks with higher delta frequencies, DF, I decided not to wait and put the new scripts out now so folks can have a play with them, praise me, curse me, etc. :)

The only thing missing, in the taskXDF.cfg file, are VRAM GB values for tasks above a DF of 0.40. Those values will need to be updated as higher DF tasks roll out (if they ever do). The script(s) now reports the average VRAM GB used for running tasks, along with individual DFs for running tasks, so when all tasks of the same DF are running, you'll know what the VRAM use is for that task DF.

As before, it's still only for Linux systems running supported AMD cards. I'm working on a Python version, and have dreams of Windows implementation, but that might be a while....

Update: Added feature to periodically report and log average task times and multiplicities. Improved formatting of reports for added clarity of data.

Thanks DF. I assume that was for running a set of tasks with DF 0.40 or 0.45?  DF values up to 0.45 have been updated in the recently uploaded taskXDF.cfg file. Glad you are getting some use out of the vramDF script. Let me know of any desired improvements.

UPDATES:
-- In the Dropbox folder, taskXDF scripts now include an extra check to not increase task X when the fraction_done value of any running task is below 5%. This is because when a task begins running, it's VRAM% may take a while to get "up to speed". Transiently low VRAM% readings during that transition period no longer cause unwanted task X increments.

In taskXDFt:
-- Commented out the new feature to report average task X and task completion times. It reports averages from 20 readings that the timed script happens to catch in the 'uploading' phase. It stores readings that it needs to average in "hidden" files, .temp_times.txt and .temp_Xs.txt, in the working directory. If that mechanism doesn't bother you and if you think it may be useful, uncomment the code block (just before the ==main body==). Results are reported intermittently in the terminal and log file along with the regular script reports.
-- Added a file size check on the taskXDF.log file to clear its contents once it reaches 100MB, just in case you love the program so much you leave it running on the timer all - the - time.  ;)
-- Fixed a coding error (a typo!) that caused erroneous task X increases.

Other:
-- Changed the file name for the timer file back to taskXDF-timer (aka, taskXDF-timer.bash in Dropbox).
-- Wrote my first Python module! So I'm about 1% along the way to Pythonification of the bash scripts and releasing a multi-OS utility.  :P

Being so encouraged that someone was using one of my scripts for monitoring O2MDF GPU tasks, I worked up another monitoring-only bash script.  The new offering reports GPU memory and boinc-client metrics, but doesn't change task multiples. (Okay, so maybe technically off-topic, but hey, close enough?)

The new scripts taskXDF-mon and taskXDF-mon-timer are available in the taskXDF-mon folder on the Dropbox page:
https://www.dropbox.com/sh/kiftk65mg59ezm6/AACeHeeKYvvrj65VawsRM107a?dl=0

Unlike the previous vramDF and vramDF-timer scripts, this new monitor does not report separate metrics for each card, but can be used with multiple card systems. What it reports are pertinent GW GPU crunching metrics that may be useful for, well...., you tell me!

Use and output details are in the README file.

Here is an example terminal output from my single card RX 5600xt host (without the pretty colors):

date     time   │ queued │ VRAM% │ GTT% │ taskGB │ X │ DFs: [running] [waiting] [ready]
----------------│--------│-------│------│--------│---│-----------------------------------
Sep 28 09:11:12 │     27 │    47 │ 1.20 │   0.70 │ 4 │ [.15 .15 .15 .15 ] [na] [.35 ]
Sep 28 09:16:12 │     27 │    55 │ 1.20 │   0.83 │ 4 │ [.15 .15 .15 .35 ] [na] [.15 ]
Sep 28 09:21:12 │     27 │    55 │ 1.20 │   0.83 │ 4 │ [.15 .35 .15 .15 ] [na] [.20 ] 

The set of Linux scripts to automatically change gravitational wave task multiples in response to AMD GPU memory requirements has been moved to GitHub, https://github.com/csecht/gravwave-taskX-df

The scripts that just monitor and report GW task memory usages and task DF values has also been moved to GitHub, https://github.com/csecht/TaskXDF-monitor

Let me know of any problems or suggestions.

Happy crunching!