All things Nvidia GPU

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3,781
Credit: 36,911,328,580
RAC: 49,185,766

mikey wrote:Stephen

mikey wrote:

Stephen "Heretic wrote:

 Hi Gary,

 . . That was my conclusion when I saw the data Archae86 referred to. No other explanation for a number that high.

 . . 140 machines??  I would have to move out to house that many. The forced ventilation sounds like a good solution but I don't think I would be comfortable working in a wind tunnel environment :).  I might try that suggestion to expand the stats graphs to more than the current 60 days. ATM I am shutting down crunching on the hot machine during the day and recommencing it at night.  There is a small A/C unit up here but it is currently kaput.  I paid an A/C guy heaps to fix it and he just topped up the gas (damned expensive gas that) but it stopped again shortly after. I think the capacitor in the condenser fan cct has gone.  Trouble is these cheap units are not designed to be readily serviced. I need to get a set of inline 240V plugs to isolate the fan so I can remove it and replace that capacitor. It is only a 1.5Kw unit but it could help.

Stephen

Would it be cheaper to replace it?

And the 'wind tunnel' effect could be reduced if you let the air in say above the door and then had the exhaust fan in the ceiling, that way most of the air flow would be up across and out not thru the space where you are. The other consideration with 140 machines is the electricity to run them, that would make for one heck on an electric bill.

depends on the system.

one one end of the spectrum you have things like RPi that use like 5W, 140 of them would use ~700W

on the other end you have extreme monolithic systems like my 7x2080Ti +EPYC CPU system that uses about 2000W by itself with GPU+CPU crunching. 140 of these would use 280kW.

 

from what I remember, Gary has mentioned that most of his systems are single GPU (a handful with 2x GPUs) mostly RX570s, with older CPUs that arent crunching. factoring in inefficiencies, I'd guess each system on average uses like 300-400W, maybe total power use around 50kW? but he's also mentioned some solar farm feeding a portion of this as well as the machines being located at a relative's business location. so it's likely some sort of industrial/commercial facility. not like he's storing these in a private house on residential power.

_________________________________________________________________________

Stephen "Heretic"
Stephen "Heretic"
Joined: 5 Feb 17
Posts: 94
Credit: 645,067,679
RAC: 0

Ian&Steve C. wrote:mikey

Ian&Steve C. wrote:

mikey wrote:

The other consideration with 140 machines is the electricity to run them, that would make for one heck on an electric bill.

depends on the system.

one one end of the spectrum you have things like RPi that use like 5W, 140 of them would use ~700W

on the other end you have extreme monolithic systems like my 7x2080Ti +EPYC CPU system that uses about 2000W by itself with GPU+CPU crunching. 140 of these would use 280kW.

 

from what I remember, Gary has mentioned that most of his systems are single GPU (a handful with 2x GPUs) mostly RX570s, with older CPUs that arent crunching. factoring in inefficiencies, I'd guess each system on average uses like 300-400W, maybe total power use around 50kW? but he's also mentioned some solar farm feeding a portion of this as well as the machines being located at a relative's business location. so it's likely some sort of industrial/commercial facility. not like he's storing these in a private house on residential power.

 . . I do not do any CPU crunching on my rigs either. I tried it on the 950 unit but it could not cope so it remains GPU only. The i5 with the 970s only has 4 cores so it is busy enough running the 2 x 970s. If I take the time to look up Petri's script, on the Ryzen unit I could probably run half a dozen CPU tasks but I  would have to spend the time testing for optimum throughput.  As it is the C2 Quad uses about 170W and is the least efficient, the Ryzen system uses about 300W and is the most efficient while the i5 uses about 425W and sits in the middle.  These figures are all with the monitors off.  But that comes close to the 1KW mark, 24 hours a day ... at 35c per KW that gets pricey enough. (close to $230 per month ... OK so just why am I doing this again?)  :)

Stephen

PS Am I correct that RPi means a Raspberry Pi? 

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3,781
Credit: 36,911,328,580
RAC: 49,185,766

is that 0.35 AUD per kWh? or

is that 0.35 AUD per kWh? or did you convert to US cents?

(35c AUD would be 26c USD)

after adding up all the nickles and dimes in terms of fees and surcharges from my power utility, my total bill comes out to be about $0.12 USD per kWh. which is about $0.16 AUD per kWh. so my bill is manageable at the moment. and coming into winter here in the northern hemisphere means I can stop running the AC for supplemental cooling, further reducing electric costs, and one of my systems is located in a place where I essentially have free electricity (but I can only have one system there, it doesn't have the power or cooling to handle more than about 1400W). If i was paying 25+ cents per kWh I'd have to seriously rethink some things.

In the peak of summer, I'll have 2-3 months of paying ~$500/mo for electric, in the dead of winter with no AC it's down to about $350/month USD

yes RPi is Raspberry Pi

_________________________________________________________________________

Stephen "Heretic"
Stephen "Heretic"
Joined: 5 Feb 17
Posts: 94
Credit: 645,067,679
RAC: 0

 . . Yep that was in AUD. But

 . . Yep that was in AUD. But to be fair it was a quick rough calculation and I used the peak daylight rates which may be a little unfair. I would have to dig out one of the accounts to calculate a more accurate average rate. It might be about the 0.25 AUD per KW as well, the off peak rate drops a bit ...  though not really that much, but it would still come down to about the 0.30 mark. In any case our bills are quarterly so every 3 months I get the love letter no one wants.  I wonder where I could go if I saved that $800 per quarter ???  Nowhere interesting unless I saved it for a long time :(

 . .  Our bills used to be quite a bit cheaper, until the sitting right wing government decided "people would benefit from privatising the power industry".  And they did, just not the people using and paying for the power.  I think quite a few of their ministers got nice cosy seats on boards, a very good retirement plan for them. (OK so I am cynical).  It happened in Victoria too, but much worse. The grid maintenance there dropped so badly that half their power network failed, and they had a very rough few weeks.  And life goes on.

Stephen

 . . It amuses me when people run ARM, RPi or whatever.  Looking at their output it would take about 100 of them to achieve the same output as one half decent PC rig. so they are actually far less power efficient.  :)

 

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5,919
Credit: 7,994,924,655
RAC: 6,263,096

My Rtx 3080ti + Gtx 1080

My Rtx 3080ti + Gtx 1080 looks like it has topped out at about 2.8 M RAC.

So I did these two commands as mentioned elsewhere in this thread.

sudo nvidia-xconfig --thermal-configuration-check --cool-bits=28 --enable-all-gpus

/usr/bin/nvidia-settings -a "[gpu:0]/GPUMemoryTransferRateOffset[4]=1000"

This processing speed has dropped some but I still haven't gotten back to below 2 minutes.  What am I missing?

 

 

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4,803
Credit: 17,876,463,120
RAC: 2,687,057

Have you rebooted since using

Have you rebooted since using the coolbits command?

Have you checked in Nvidia X Server Settings panel that those changes have in affect occurred?

Is your memory clock shown for Level 3 19500Mhz?

 

Stephen "Heretic"
Stephen "Heretic"
Joined: 5 Feb 17
Posts: 94
Credit: 645,067,679
RAC: 0

Does anyone remember how to

Does anyone remember how to disable the screensaver when running BOINC?

Stephen

 

Harri Liljeroos
Harri Liljeroos
Joined: 10 Dec 05
Posts: 3,860
Credit: 2,979,567,856
RAC: 979,912

If you are using Windows, you

If you are using Windows, you do it from Windows settings (Control Panel).

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5,919
Credit: 7,994,924,655
RAC: 6,263,096

Keith Myers wrote: Have you

Keith Myers wrote:

Have you rebooted since using the coolbits command?

Have you checked in NVIDIA X Server Settings panel that those changes have in affect occurred?

Is your memory clock shown for Level 3 19500Mhz?

Rebooted.

Server Panel is exposing the ability to offset -/+

With a memory offset of 3200+ the Panel is claiming: 21702

I got below 2 minutes but I had gpu speed 100+ when it crashed.

Trying 3200+ with no gpu speed off set.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5,919
Credit: 7,994,924,655
RAC: 6,263,096

Overclocking an Gtx

Overclocking an Gtx 1080?

So far even overclock memory to 200+ seems to generate some computation errors which cause the server to slow down the update cycle.

Does anyone have any guidance/ideas?

I am still working on maximizing my RAC for my 3080 ti / 1080 combo.

About the only other things I can now afford would be a 3080 ti + Rx 5700 which would require Ian&SteveC's coding to allow the Rx to run two instead of one task.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.