Thanks, Ian&Steve C, for sharing your insights. Given that the platform (MacIntel vs PC) and connectivity differences (PCie vs TB3) appear to have only a modest impact on the E@H productivity of a typical GPU, I wonder if it would be nice for the users to start posting E@H productivity data for different GPU models? Or are contemporary data already available? I would be willing to share the E@H data for my GPUs (RX Vega 56s, RX Vega 64s, RX 5700 XTs, RX 6600s, RX 6600 XTs, and RX 6800 XTs). This would help users interested in setting up a E@H cruncher/farm do a cost/productivity analysis.
Alternatively, I wonder how hard it would be to write some R or Python code to extract the E@H data on a project-wide level? Unfortunately, that is beyond my current coding capabilities.
"I was born in a small town, and I live in a small town." - John Mellencamp
Thanks for sharing your insights, Locations Unknown. In my experience, **any** farm with 13 GPUs is a challenge to operate 24/7, regardless of how they are distributed across rigs. Moreover, I sympathize with your challenge of supplying enough power to your crunchers. I had to upgrade my home office to a 30 amp circuit to keep my crunchers from tripping the 20 amp breaker.
"I was born in a small town, and I live in a small town." - John Mellencamp
Could you possibly share with the dumb users like me what are the differences between PCIe vs TB3?
I do know (or I think I know) that the PCIe is for computers (either Windows or Linux) and TB3 is (I think) for Macs. And that all I know, or think I know. :)
Article is a bit out of date though since there is USB4 now that matches Thunderbolt 4 at 40 Gbs.
All the same physical connector either way.
Bandwidth is big enough to connect video devices like cards and displays so good for external devices like gpus housed in external housings or monitors.
That article was an enlightenment for me. I've been just casually watching what Thunderbolt was, but couldn't connect it to TB for some reason. DOH! Plus, I was unaware of the Thunderbolt and USB-C ports essentially being interchangeable, but with so many variables.
TB3 = Thunderbolt 3. it's a special kind of USB-C connection that can carry PCIe lanes over the USB-C interface. it can carry up to 4x PCIe 3.0 lanes. PCIe is PCIe is PCIe. it works for any computer and is supported by all OS's, mac included.
there's a little more to it but that's the quick and dirty answer, Keith's link goes into more detail.
Thunderbolt 3 (and 4) have been commonly used to hook up external GPU docks for laptops and macs. That's what Dave is running.
I now have a much better understanding of what Thunderbolt 3 (& 4) and USB-C is, and how similar they are. I was not aware that they used the same port, but for many different reasons.
I originally loaded up on RX 6600 XT's for mining crypto currency, running Boinc was not in my original consideration. For mining crypto, 6600 XT's are one of the most efficient cards on the market. I used to run a bunch of 580's, one of the most widely used GPU's to mine with back in the day, and the 6600 XT would mine the same amount of coin for half the wattage. I've mined with AMD 6600's, 6700 XT's, 6800's, Nvidia 3060, 3060 TI, 3070, 3080, 3080 TI, and the 3090... The only GPU I found to roughly match the efficiency was maybe the 3070 and possibly the 3060 TI. (That is hash rate per watt) I had the 6600 XT's perfectly underclocked to mine Ethereum, running at about 50 to 60 watts per card.
For E&H, I have my 6600 XT's underclocked, running at about 65 watts. With that decreased load, I'm still completing E&H tasks about every 12 minutes. I'm upgrading my 6600 XT's this weekend to newer AMD Ryzen CPU's which should help complete tasks faster.
I don't have an electrical engineering background, so when I got into mining there was a lot I needed to learn before safely running as many GPU's as I had in my basement. I ended up installing a bunch dedicated 20 AMP breakers, each one with its own dedicated 4 outlet box, with 12 gauge wire in the wall. I then set up wireless power meters on every outlet to track my power consumption. Since I only used 15 AMP standard outlets, I had to be mindful of how much juice I was running on each individual PSU.
Each duplex outlet could handle a max load of 15 amps or 1800 watts. Since I was running 24/7, I dropped my limit per duplex down to about 1400 watts (-20%). I then had to make sure I wasn't hitting my max load on the breakers, which is about 2400 watts, and again I lowered my continuous max load by about 20% or 1900 watts. I also invested in some industrial level power strips that had added fault protections and had at least 2 or 3 specialized fire extinguisher for electrical fires at arms length incase anything burst into flames.
If I could do it all over again, the only change I would make would be to install dedicated 240 volt outlets instead of 110 volt standard outlets. For mining, if I was running 15 amp 240 volt outlets, I could probably safely run 2800 watts. So I could have crammed more GPU's into my basement.
It took a while but I eventually had a pretty stable home mining operation going in my basement for several years. I had so much fun building the rigs and maintaining everything. When the crypto market crashed and ETH went to proof of stake, I made the decision to keep everything current gen and sell off all my old GPU's like 580's & 1000 series Nvidia cards. I then had around 30 GPU's just sitting idle and figured, what the hell, I might as well point them at a Boinc project to help offset my mining income from the first half of the year. Plus, I'll be contributing to Science.
Here is my End of the Year Report on standing up a Farm.
I now have 6 systems running Boinc research. I started with two.
2 Epyc-based
4 Ryzen-based, one of which is running windows. 2x 3950x, 2x 3700x
I am running 4 distinct GPU crunching systems. 2x Rtx 3080 ti gpus on one system. One each, rtx 3080 ti, gtx 1080 ti and gtx 1060 6GB
Most of my systems are processing E@H, U@H and WCG. Fall back projects also include Gpugrid, Milkyway and PrimeGrid. But on some systems either WCG is primary with u@h as 0 resource backup and vice versa. On the box with the gtx 1080 ti I am trying to run the WCG GPU task as primary but the zero resource e@h GPU tasks are getting plenty of action.
I will continue tinkering (of course) but I doubt I will be running more than 6 systems.
I may start shutting some of them down. Especially as warmer weather hits this spring.
Tom M
A Proud member of the O.F.A. (Old Farts Association). Be well, do good work, and keep in touch.® (Garrison Keillor) I want some more patience. RIGHT NOW!
Thanks, Ian&Steve C, for
)
Thanks, Ian&Steve C, for sharing your insights. Given that the platform (MacIntel vs PC) and connectivity differences (PCie vs TB3) appear to have only a modest impact on the E@H productivity of a typical GPU, I wonder if it would be nice for the users to start posting E@H productivity data for different GPU models? Or are contemporary data already available? I would be willing to share the E@H data for my GPUs (RX Vega 56s, RX Vega 64s, RX 5700 XTs, RX 6600s, RX 6600 XTs, and RX 6800 XTs). This would help users interested in setting up a E@H cruncher/farm do a cost/productivity analysis.
Alternatively, I wonder how hard it would be to write some R or Python code to extract the E@H data on a project-wide level? Unfortunately, that is beyond my current coding capabilities.
"I was born in a small town, and I live in a small town." - John Mellencamp
Thanks for sharing your
)
Thanks for sharing your insights, Locations Unknown. In my experience, **any** farm with 13 GPUs is a challenge to operate 24/7, regardless of how they are distributed across rigs. Moreover, I sympathize with your challenge of supplying enough power to your crunchers. I had to upgrade my home office to a 30 amp circuit to keep my crunchers from tripping the 20 amp breaker.
"I was born in a small town, and I live in a small town." - John Mellencamp
Ian & Tigers_Dave, Could
)
Ian & Tigers_Dave,
Could you possibly share with the dumb users like me what are the differences between PCIe vs TB3?
I do know (or I think I know) that the PCIe is for computers (either Windows or Linux) and TB3 is (I think) for Macs. And that all I know, or think I know. :)
Proud member of the Old Farts Association
Here you go
)
Here you go George.
Thunderbolt vs. USB-C: What's the Difference?
Article is a bit out of date though since there is USB4 now that matches Thunderbolt 4 at 40 Gbs.
All the same physical connector either way.
Bandwidth is big enough to connect video devices like cards and displays so good for external devices like gpus housed in external housings or monitors.
Thanks Keith, That article
)
Thanks Keith,
That article was an enlightenment for me. I've been just casually watching what Thunderbolt was, but couldn't connect it to TB for some reason. DOH! Plus, I was unaware of the Thunderbolt and USB-C ports essentially being interchangeable, but with so many variables.
Proud member of the Old Farts Association
TB3 = Thunderbolt 3. it's a
)
TB3 = Thunderbolt 3. it's a special kind of USB-C connection that can carry PCIe lanes over the USB-C interface. it can carry up to 4x PCIe 3.0 lanes. PCIe is PCIe is PCIe. it works for any computer and is supported by all OS's, mac included.
there's a little more to it but that's the quick and dirty answer, Keith's link goes into more detail.
Thunderbolt 3 (and 4) have been commonly used to hook up external GPU docks for laptops and macs. That's what Dave is running.
_________________________________________________________________________
Thanks Ian, I now have a
)
Thanks Ian,
I now have a much better understanding of what Thunderbolt 3 (& 4) and USB-C is, and how similar they are. I was not aware that they used the same port, but for many different reasons.
Proud member of the Old Farts Association
I originally loaded up on RX
)
I originally loaded up on RX 6600 XT's for mining crypto currency, running Boinc was not in my original consideration. For mining crypto, 6600 XT's are one of the most efficient cards on the market. I used to run a bunch of 580's, one of the most widely used GPU's to mine with back in the day, and the 6600 XT would mine the same amount of coin for half the wattage. I've mined with AMD 6600's, 6700 XT's, 6800's, Nvidia 3060, 3060 TI, 3070, 3080, 3080 TI, and the 3090... The only GPU I found to roughly match the efficiency was maybe the 3070 and possibly the 3060 TI. (That is hash rate per watt) I had the 6600 XT's perfectly underclocked to mine Ethereum, running at about 50 to 60 watts per card.
For E&H, I have my 6600 XT's underclocked, running at about 65 watts. With that decreased load, I'm still completing E&H tasks about every 12 minutes. I'm upgrading my 6600 XT's this weekend to newer AMD Ryzen CPU's which should help complete tasks faster.
I don't have an electrical
)
I don't have an electrical engineering background, so when I got into mining there was a lot I needed to learn before safely running as many GPU's as I had in my basement. I ended up installing a bunch dedicated 20 AMP breakers, each one with its own dedicated 4 outlet box, with 12 gauge wire in the wall. I then set up wireless power meters on every outlet to track my power consumption. Since I only used 15 AMP standard outlets, I had to be mindful of how much juice I was running on each individual PSU.
Each duplex outlet could handle a max load of 15 amps or 1800 watts. Since I was running 24/7, I dropped my limit per duplex down to about 1400 watts (-20%). I then had to make sure I wasn't hitting my max load on the breakers, which is about 2400 watts, and again I lowered my continuous max load by about 20% or 1900 watts. I also invested in some industrial level power strips that had added fault protections and had at least 2 or 3 specialized fire extinguisher for electrical fires at arms length incase anything burst into flames.
If I could do it all over again, the only change I would make would be to install dedicated 240 volt outlets instead of 110 volt standard outlets. For mining, if I was running 15 amp 240 volt outlets, I could probably safely run 2800 watts. So I could have crammed more GPU's into my basement.
It took a while but I eventually had a pretty stable home mining operation going in my basement for several years. I had so much fun building the rigs and maintaining everything. When the crypto market crashed and ETH went to proof of stake, I made the decision to keep everything current gen and sell off all my old GPU's like 580's & 1000 series Nvidia cards. I then had around 30 GPU's just sitting idle and figured, what the hell, I might as well point them at a Boinc project to help offset my mining income from the first half of the year. Plus, I'll be contributing to Science.
Here is my End of the Year
)
Here is my End of the Year Report on standing up a Farm.
I now have 6 systems running Boinc research. I started with two.
2 Epyc-based
4 Ryzen-based, one of which is running windows. 2x 3950x, 2x 3700x
I am running 4 distinct GPU crunching systems. 2x Rtx 3080 ti gpus on one system. One each, rtx 3080 ti, gtx 1080 ti and gtx 1060 6GB
Most of my systems are processing E@H, U@H and WCG. Fall back projects also include Gpugrid, Milkyway and PrimeGrid. But on some systems either WCG is primary with u@h as 0 resource backup and vice versa. On the box with the gtx 1080 ti I am trying to run the WCG GPU task as primary but the zero resource e@h GPU tasks are getting plenty of action.
I will continue tinkering (of course) but I doubt I will be running more than 6 systems.
I may start shutting some of them down. Especially as warmer weather hits this spring.
Tom M
A Proud member of the O.F.A. (Old Farts Association). Be well, do good work, and keep in touch.® (Garrison Keillor) I want some more patience. RIGHT NOW!