GPU Crunching - What setup works 'best', what does it cost, what comparison metric is best to use?

archae86
archae86
Joined: 6 Dec 05
Posts: 3,157
Credit: 7,183,194,931
RAC: 779,698

Configuration 2 - Hardware

Configuration 2 - Hardware machine name is Stoll7
[pre]
CPU: Intel core i5 2500K Sandy Bridge @ 3.3 GHz $220
Motherboard: Asrock Z77 Extreme4 130
GPU1: Gigabyte GTX660 190
GPU2: EVGA GTX 750Ti 02G-P4-3753-KR 140
RAM: 2 x 4GB DDR3 1600 (PC3 12800) F3-12800CL9D-8GBRL 63
Boot SSD: Crucial M4 CT064M4SSD2 64GB 86
Boot HD: WD20EZ4X 2TB hard drive 80
PSU: Seasonic X650 Gold (SS-650KM) 110
----
Total hardware price for the agreed items is 1019
----
[/pre]
Configuration 2 - Software
[pre]
OS: Microsoft Windows 7 Home Premium x64 Edition 95
Driver: 344.60
BOINC Version: 7.4.36 64bit
Total software cost is $95
[/pre]
Daily hardware and software purchase cost
For the assumed 4 year life of the components, the figure is (1019+95)/1461 = $US0.762/day.

Power Consumption, Productivity and Running costs - Power is assumed at $US0.25/unit (kWH)
[pre]
Daily Power Use Task Mix Av Secs per Task Tasks per Day Credit/Day
Hardware Setup and Running State W kWH $US CPU,GPU1,GPU2 CPU GPU1 GPU2 CPU GPU1 GPU2 CPU GPU1 GPU2 Calculated Credit/kWH Calculated Credit/$US
================================ === ===== ===== ============= ===================== ================= ================ ===================== =====================
System at idle - GPU installed: 73 1.752 0.438
Crunching 2xFGRP4 + (2+2)xBRP6: 251 6.020 1.505 2 2 2 24,180 11,680 15,870 7.14 10.89 14.80 4952 47909 65096 117958/6.020 = 19.593 117958/(0.762+1.50) = 52,019

[/pre]
Notes
1. This system uses the magic Intel software to accelerate much disk access using an SSD, so to price properly the boot drive function, I've included both
2. However this scheme, which considerably aids application launch time, is probably of negligible contribution to BOINC output, so people should discount this cost.
3. The CPU is no longer readily available, in such cases, I used the most recent NewEgg price as retained by Google search.
4. In the case (the SSD) where the component is available new, but at a preposterous price, I used the actual purchase price, and would coach people to find another unit, or do without.

Comments

  • * For the crunching config noted above,

Purchase cost = $US0.76/day and Running cost = $US1.50/day
* This config produces 52.0k credits/$US. assuming $US0.25/kWHr electric power, and four year replacement cycle
* The purchase cost for each 1K credit/day of productivity is Purchase cost/1k credit/day = $US9.44
* Clock rates and timings are stock for both GPUs, CPU, and RAM
* I use Process Lasso priority affinity and priority adjustment to try to give low latency CPU service to the GPU tasks.
* The BRP6 results above were materially aided by the improvements of version 1.52. As both GPUs are nvidia, it could be that the promised forthcoming higher CUDA version release will help further.
* When I was running the GTX660 alone on this machine, with an earlier version of software, the multiplicity I chose was 3. I suspect that with the lower CPU (and presumably PCI bus) usage of 1.52 that the optimum multiplicity might have changed, but I don't intend to explore this until a higher CUDA version is released.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5,870
Credit: 116,079,768,920
RAC: 35,936,914

RE: Comments * For the

Quote:
Comments
  • * For the crunching config noted above,
Purchase cost = $US0.76/day

and Running cost = $US1.50/day
* This config produces 52.0k credits/$US. assuming $US0.25/kWHr electric power, and four year replacement cycle
* The purchase cost for each 1K credit/day of productivity is Purchase cost/1k credit/day = $US9.44
* ...

Peter,

Thanks for publishing this second configuration. Thanks also for laying out the calculation of your 'purchase cost productivity'. I was (and probably still am) having a severe mental block about it. I fully admit that I'm struggling to understand what this is really telling me. OK, it is saying that the equipment (and OS) that you have purchased, has cost you $9.44 for every 1k of credit per day that you will receive (1114/117.958 = $9.44) but I'm really not used to thinking that way, and that is probably my problem.

To me, it's more meaningful to know that, per day, the machine costs $0.76 to own and $1.50 to run and for that total expenditure commitment, I'll get 52k credit per dollar. When I spend a lot of money, I like to know I'm getting value for the expenditure. My (possibly naive) thinking is that if I maximise the credit/dollar figure (for the 'science' that interests me) I'm getting 'better' value for money.

I hasten to add that there is no problem for me in people showing their data in whichever way they like. It's actually quite mentally stimulating to be forced to think through different ways of presentation. Both you and I (and also some others) have spent quite a bit of time on this. I'd really like to know what the general readership thinks of what has been presented so far. I have one more post to make about efficiency of PSUs and by that time I hope others may have told us what they think.

Cheers,
Gary.

archae86
archae86
Joined: 6 Dec 05
Posts: 3,157
Credit: 7,183,194,931
RAC: 779,698

RE: Thanks also for laying

Quote:
Thanks also for laying out the calculation of your 'purchase cost productivity'. I was (and probably still am) having a severe mental block about it. I fully admit that I'm struggling to understand what this is really telling me.


It is just unhinging purchase cost from a particular assumption of replacement lifetime. There is, to my mind, a direct analogy here to unhinging power consumption from the assumed local price of incremental power. Posting the two primary pieces makes it one step simpler for people to apply their own preferred assumptions for these two things (power price and replacement lifetime).

As to the choice of presenting it, vs. the inverse, they contain exactly the same information, so it is just a question of which people find instinctively more comfortable.

Lastly, as you probably noticed, this machine had two dissimilar GPUs, so I had a try at extending your format for that case.

While my specific case will be uncommon, there will be other people with hosts with two GPUs where one of the GPUs is an Intel one onboard the CPU chip. This may even get common in the future as more user hosts have modern Intel chips, and, perhaps, eventually the Intel drivers and the Einstein code make better music together than currently.

AgentB
AgentB
Joined: 17 Mar 12
Posts: 915
Credit: 513,211,304
RAC: 0

HD7990 New Build - Standard

HD7990 New Build - Standard power and costs 4 year lifetime

machine name is tauon

[pre]
NewEgg List Actual Decription
USD USD USD
Case 64 80 19 Antec Three Hundred
Fans 41 70 0 5 Antec Case fans
Mobo 139 160 170 Gigabyte Z97X-UD3H-BK Intel LGA1150 Z97 ATX
CPU 224 250 265 Intel Core i5 i5-4690K
Cooler 43 53 25 Raijintek EreBoss High Performance CPU Cooler
SSD 98 110 82 Samsung 2.5-Inch 120 GB 850 EVO
DVD 20 20 0 SAMSUNG DVD Writer24X Model SH-224FB/BSBE
Memory 127 140 167 Corsair Vengeance (2x8GB) DDR3 1866 Mhz
PSU 149 170 121 Corsair HX750i
GPU 900 1000 256 XFX Radeon HD 7990 (6144 MB) (FX799AXNF9)
KB+M 15 25 0 Logitech MK120 Wired USB Keyboard and Mouse
OS++ 0 0 0 Ubuntu 14.04 LTS
Monitor 89 129 0 Samsung SC200 Series S19C200NY

Total 1909 2207 1106

Life 4 Lifetime
Setup/Y 477 552 276 Cost per year
Setup/D 1.31 1.51 0.76 Cost per day
================================


kWHr/D 10.32 430W
kWHr/Y 3767

Power/Y 942 942 942 power at 0.25 USD/kWhr
Power/D 2.58 2.58 2.58
================================

Total/Y 1419 1493 1218
Total/D 3.89 4.09 3.34
================================

Cred/D 276346 276346 276346 3+3 BRP6 (average time 8254s)
Cred/$ 71085 67542 82806

[/pre]

Notes
1. Apologies for the poor formatting it is from a spreadsheet
2. I am having some trouble getting this thing stable, the 7990 is two GPU sharing the one card, and it gets very hot quickly. I thought i had it stable running standard clocks for a few days then the first sign of warm weather it became very unruly. It is running at reduced clocks at the moment. I will be a little disappointed if the RAC is less than 250K for this when stable running 24x7.
3. The 7990 is no longer selling, and i suspect the reason is cooling issues, so I´ve worked from old prices. I picked this GPU secondhand.
4. There is no way i could justify this build buying new, and i have put what i actually paid. The mobo, cpu, memory, SSD, cooler and PSU all were new, the rest secondhand or spares.
5. Most of the new equipment is good quality, and i would expect this system to last 5-7 yrs.
6. Notice how i have to pay higher than New Egg list price, for some equipment, that is not by choice and is using official exchange rates!
7. My actual power costs are much lower, and i have 7 hours / day at half price, so i have some options to run this at lower cost. Power costs are incremental.
8. My power meter values need work (430W feels low) and about 90W idle with fans pumping) but are probably accurate withing 10%. HX750i is digital power supply so once i work out the Corsair plus plus open-source software - i should get accurate readings direct from the power supply.
9. For piece of mind, mainly my pocket, i have used my values of system life, and power costs below for 24 hour running and off peak only running
10. Apologies again for the terse formatting.

HD7990 New Build 7x24 - Local power costs and 5 yr lifetime

[pre]
Life 5 Lifetime
Setup/Y 382 441 221 Cost per year
Setup/D 1.05 1.21 0.61 Cost per day
================================

kWHr/D 10.32 430W
kWHr/Y 3767

Power/Y 591 591 591 power at 0.18/0.09 USD/kWhr
Power/D 1.62 1.62 1.62
================================

Total/Y 973 1032 812
Total/D 2.66 2.83 2.22
================================

Cred/D 276346 276346 276346 3+3 BRP6 (average time 8254s)
Cred/$ 103703 97720 124225

[/pre]

HD7990 New Build 7x7 (off peak) - Local power costs and 5 yr lifetime

[pre]
Life 5 Lifetime
Setup/Y 382 441 221 Cost per year
Setup/D 1.05 1.21 0.61 Cost per day
================================

kWHr/D 3.01 430W
kWHr/Y 1099

Power/Y 101 101 101 power at 0.18/0.09 USD/kWhr
Power/D 0.28 0.28 0.28
================================

Total/Y 483 542 322
Total/D 1.32 1.49 0.88
================================

Cred/D 80601 80601 80601 3+3 BRP6 (average time 8254s)
Cred/$ 60951 54256 91366
[/pre]

archae86
archae86
Joined: 6 Dec 05
Posts: 3,157
Credit: 7,183,194,931
RAC: 779,698

@AgentB First and

@AgentB

First and foremost, thanks for posting. This exercise needs to extend beyond a private conversation between just a few of us to be more useful to the hoped-for audience.

Still, I'll pose some questions and make some comments.

1. Why do you distrust the 430W system power consumption? What sort of measuring method does it come from, and why would you trust readings reported by the power supply more?

2. If you don't mind my asking, where do you live, with power prices much lower than US$0.25/kWHr, but component prices far above NewEgg prices? I understand that could describe many places--just curious of your case?

Comments:

The reason to use NewEgg prices is not to figure out what you personally paid, but rather to provide a level platform for comparison to help guide other users in selection choices. It really does not help them to know what you really paid, or reused.

The purpose in pricing a standard subset of total parts is again to help other users make comparisons. Adding in other components hurts the comparison purpose.

The lifetime question, for which I suggested 3 years but accepted Gary's 4 years is not meant to be a prediction of failure time, but an average practical replacement interval. Looking back in time, equipment much older than that is difficult to justify running as a system for the pure problem of the cost of power. Even with free purchase cost, a donated 6 year old system at this point would not compete with a retail priced new purchase of an efficient system. People who don't pay for their power and don't want to consider the consequences of consuming it may well have a different opinion.

KLiK
KLiK
Joined: 1 Apr 14
Posts: 57
Credit: 374,210,327
RAC: 1,477,039

Check out this power of

AgentB
AgentB
Joined: 17 Mar 12
Posts: 915
Credit: 513,211,304
RAC: 0

Thanks for looking over the

Thanks for looking over the numbers

Quote:

1. Why do you distrust the 430W system power consumption? What sort of measuring method does it come from, and why would you trust readings reported by the power supply more?

I just lack a little confidence of the accuracy, and it is this number which is the VIP one.

First problem is the actual meter readings, from an older SilverCrest meter - for example it would show 93, 98, 102, 94, 103, 89 which i have to take a guess at 97. I don´t understand why it fluctuates and the 430 is a guess at a average or the maximum. Seeing the PSU info might make it clearer/give confidence, and get it logged into a table. There is a discussion on a later Silvercrest model manual i found, about the power factor ¨cos phi¨(?) which is 0.85 displayed on the meter, where they say 1.0 is ideal and less causes the readings to be unstable. The total function might be better, to get a value i would need to run for quite a few hours as the total is in KWhr. This means it stays stable for that time, and this meter has a habit of needing resetting every now and then.

Second my rule of thumb says the card should use 450 itself at peak and the CPU + mobo + PSU must be at least 150, so say for 80% of that say theoretical 600W gets to 480W.

Thirdly - it´s gives a lot of heat and i´m comparing it to known radiator!

Quote:

2. If you don't mind my asking, where do you live, with power prices much lower than US$0.25/kWHr, but component prices far above NewEgg prices? I understand that could describe many places--just curious of your case?

UK, sorry i should have mentioned or put it in my profile. There is quite a variety of tariff options with choice of supplier (reseller) and plans. To my surprise this morning, I received next years prices, and yes they went up! I may have to shop around for a new supplier or sell the 7990.

It´s worth looking here for a rough table on world electricity prices from 1 to 50c / KWHr seems the typical range!

Quote:

The reason to use NewEgg prices is not to figure out what you personally paid, but rather to provide a level platform for comparison to help guide other users in selection choices. It really does not help them to know what you really paid, or reused.

Understood and wholeheartedly agree, if you are comparing component prices a standard point of reference is needed, and i certainly didn´t want to suggest others do so, but just wanted to highlight the typical variability in costs (in both directions and in both categories) with a real life example outside of the standard assumptions.

I thought that might help, see the variability in prices. I am sure others further afield will see different prices.

Quote:

The purpose in pricing a standard subset of total parts is again to help other users make comparisons. Adding in other components hurts the comparison purpose.

I can adjust the list to remove what non-standard components, and redo the numbers if that helps, i just listed what i needed to set it up and run a real example.

Quote:

The lifetime question, for which I suggested 3 years but accepted Gary's 4 years is not meant to be a prediction of failure time, but an average practical replacement interval.

This is an interesting question and i misunderstood the term, the practical lifetime, as it will differ from place to place i guess. I guess you mean economically viable? If power costs are high and new, efficient equipment cheap, it will be shorter. If the reverse, then lifetimes will be longer.

archae86
archae86
Joined: 6 Dec 05
Posts: 3,157
Credit: 7,183,194,931
RAC: 779,698

AgentB wrote:First problem is

AgentB wrote:
First problem is the actual meter readings, from an older SilverCrest meter - for example it would show 93, 98, 102, 94, 103, 89 which i have to take a guess at 97. I don´t understand why it fluctuates and the 430 is a guess at a average or the maximum. Seeing the PSU info might make it clearer/give confidence, and get it logged into a table. There is a discussion on a later Silvercrest model manual i found, about the power factor ¨cos phi¨(?) which is 0.85 displayed on the meter, where they say 1.0 is ideal and less causes the readings to be unstable. The total function might be better, to get a value i would need to run for quite a few hours as the total is in KWhr. This means it stays stable for that time, and this meter has a habit of needing resetting every now and then.

Power factor refers to a timing offset of the voltage and current waveforms, generally caused by the device not being a purely resistive load. In principle, if you just attached a power resistor to the power line, the meter should show a power factor of 1.00. Inductive or capacitive aspects of the load push it away from 1.

But I think the number you see fluctuates because the real power consumption fluctuates. I've observed this on meters (and UPS power consumption indicators) on my own desktop PCs and see it to be strongly application dependent--some are very stable, and some have serious variation.

This is the reason I strongly advocate using an available averaging function, usually provided in the form of energy consumption over a period of time (kWHr). If you are fortunate the meter provides enough resolution to do this in a reasonable amount of time--otherwise it is not practical.

I think Gary is still warping this ship toward a standardized format--so I'm not asking a revision from you, I was just using your posted example to articulate some of the considerations as I see them.

Thanks again for taking the time to post and contributing to the discussion of considerations.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.