Standing up BOINC end-user "Farms"

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674049476
RAC: 1789008
Topic 228540

Hi,

It occurred to me that a thread on setting up multiple slow(er) "farms" like various participants have done might be an interesting topic.

I will note that a "Farm" is a TBD limitation here.  But I will claim that two systems may not qualify. :)

Some of our participants don't have top-performing individual systems but have a massive presence because of many individual modest systems (Yes, Gary I am thinking of you).

So given our current "climate" what kind of system sizes/brands/models would you pick?

What kind of money would you invest per system?  Would you crunch with gpus at all?

Noisy Tom M

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674049476
RAC: 1789008

Gary, Tiger_Dave, Freewill,

Gary, Tiger_Dave, Freewill, Keith?

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674049476
RAC: 1789008

I am thinking about the

I am thinking about the previous gen AMD systems like a Ryzen 3700x (8c/16t) on something like a b450 f motherboard.  2 8-16 GB ram cards.  Linux/Ubuntu in some flavor.

Anything from a gt 710 (for a CPU-only system) to a gtx 1060 to any low-cost AMD/Nvidia GPU that is late model enough to crunch brp7 tasks.

I have the ram and motherboard(s).  I would need a few more 3700x/air cooler kits.  And a selection of low-cost gpus.  Would I jump on a working Radeon VII for $100?  Of course!

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tigers_Dave
Tigers_Dave
Joined: 25 Mar 09
Posts: 228
Credit: 9248626727
RAC: 0

I am flattered to be asked,

I am flattered to be asked, Tom.  Note that I am a professor at a relatively large (~30,000 students) US university, where my research team investigates mechanisms of metastatic melanoma (skin cancer), triple-negative breast cancer, and childhood brain tumors.  We also seek mechanism-based strategies for treating these cancers.  Thus, many of my computers are connected to scientific equipment and/or run specialized scientific software.  For historical and practical reasons, all of my work computers that are connected to the internet (and therefore are available to BOINC) are Macs.  Almost all of my work computers, monitors, GPUs, printers, and other peripherals were purchased with my own personal funds, not university funds nor research grant funds. Yes, my wife is a saint.  

At this moment, roughly 98% of my GPU capacity is committed to Einstein - I crunch for Amicable Numbers only when I can't get sufficient work from E@H.  100% of my CPU capacity is committed to Rosetta.

To summarize, my collection of crunchers should be considered an exemplar of what NOT to do in order to maximize return on hardware investment and electrical consumption.  My setups are pretty fluid, as most of my GPUs were purchased used and fail at a rather high frequency.

- six 2008-2010 MacPro 3,1, 4,1, and 5,1 ("cheese graters") computers with dual quad-core or dual hex-core Xeon processors and a variety of internal GPUs (AMD RX 580 and NVIDIA GTX 680, 980, 980Ti, and 1080Ti).  Parenthetically, several of these computers were at one time connected to Thunderbolt 3 eGPUs using a PCI card.  These setups were unstable and I abandoned this paradigm.  These machines average 50k - 500k credits per day.  Estimated average hardware replacement cost for each of these computers (excluding monitors and other peripherals): $700

- seven 2018 MacMini 8,1 computers with a quad-core i3 or hex-core i5 processor.  Each is connected to 3-4 AMD GPUs (a variety of RX 580, RX Vega 56, RX Vega 64, RX 5700 XT, RX 6600 XT, RX 6800 XT) housed in individual Thunderbolt 3 eGPU enclosures (Akitio or Sonnet).  These machines average 1800k-2500k credits per day, depending on the number and mix of eGPUs.  Estimated average hardware replacement cost for each of these seven setups (excluding monitors and other peripherals): $2400

- one 2019 MacBook Pro 16,1 computer with an 8-core i9 processor.  It is connected via TB3 to RX Vega 64 and RX 580 eGPUs.  It averages 1400k credits/day.  Estimated hardware replacement cost (excluding monitors and other peripherals): $2000

Total estimated hardware replacement cost for these 14 rigs: $23,000

My steady-state productivity is roughly 17M credits/day.  So, I am getting roughly 700 credits/$ of hardware replacement cost. So, as I said earlier, my collection of crunchers should be considered an exemplar of what NOT to do in order to maximize return on hardware investment.  And, again, my wife is a saint for indulging me in this manner.

If I was going to build a computer for Einstein/Rosetta crunching at home and not for any of my professional activities, I would build a Linux box with multiple internal NVIDIA GPUs.  

"I was born in a small town, and I live in a small town." - John Mellencamp

Tigers_Dave
Tigers_Dave
Joined: 25 Mar 09
Posts: 228
Credit: 9248626727
RAC: 0

Some of you may wonder why my

Some of you may wonder why my MacMini and MacBook Pro systems are not connected to NVIDIA eGPUs.  Unfortunately, recent versions of MacOS are not compatible with NVIDIA eGPUs.

Likewise, some of you may wonder why I don't run Linux or Windows on my Macs.  It is because some of my scientific and productivity software here at work doesn't work under Linux and my students and I don't wish to implement dual-boot setups.  You absolutely have the right to think that we are being foolish.

Furthermore, if I started running Windows on my Macs, my IT office would insist on administering those computers.  That is not acceptable to me.  Again, you absolutely have the right to think that philosophy is foolish.

"I was born in a small town, and I live in a small town." - John Mellencamp

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674049476
RAC: 1789008

Tigers_Dave

Tigers_Dave wrote:

Furthermore, if I started running Windows on my Macs, my IT office would insist on administering those computers.  That is not acceptable to me.  Again, you absolutely have the right to think that philosophy is foolish.

While many do run Windows around here I would be surprised if any Windows machines were current top rankers.  But for standing up a small Farm with modest hardware, if you already had a windows box it would be VERY tempting to drop in a discrete GPU and install Boinc.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

mikey
mikey
Joined: 22 Jan 05
Posts: 11889
Credit: 1828108445
RAC: 206129

Tom M wrote: Hi, It

Tom M wrote:

Hi,

It occurred to me that a thread on setting up multiple slow(er) "farms" like various participants have done might be an interesting topic.

I will note that a "Farm" is a TBD limitation here.  But I will claim that two systems may not qualify. :)

Noisy Tom M 

MANY years ago someone on Seti said a 'farm' was just below a 'ranch' and that it meant more than 10 boxes, I'm not sure that 'box' thing is valid anymore with more and more cpu's out there in the 32 and up range.

That being said I have 17 boxes running Boinc right now and all but 1 has a gpu crunching some project based on my goals and it's capabilities. About half of my boxes are Windows OS's ranging from Win7 to the latest Win11 with the other half of the boxes being various versions of Linux Mint. Unfortunately some projects only run on some OS's and not on others so I'm going to keep the Windows boxes around for a long time is required for my goals. I also have a 'cheese grater' Mac that's now running Linux as it crunches faster and is easier to manage on a day to day basis for me.

I started out on Windows pc's back in the late 70's but didn't start crunching until the late 80's and have never stopped crunching since. MOST of my boxes are 10+ years old from their original purchase date and then I bought them 'off lease' as inexpensive boxes only needing a crunching gpu and a harddrive to be up and running. Buying dual Xeon quad core cpu's that do do HT already installed in a pc case  with 16gb of ram for $139 was a VERY good deal!!!

GWGeorge007
GWGeorge007
Joined: 8 Jan 18
Posts: 2767
Credit: 4541011081
RAC: 2153348

Tom M wrote:Tigers_Dave

Tom M wrote:

Tigers_Dave wrote:

Furthermore, if I started running Windows on my Macs, my IT office would insist on administering those computers.  That is not acceptable to me.  Again, you absolutely have the right to think that philosophy is foolish.

While many do run Windows around here I would be surprised if any Windows machines were current top rankers.  But for standing up a small Farm with modest hardware, if you already had a windows box it would be VERY tempting to drop in a discrete GPU and install Boinc.

Careful Tom, you may be surprised to know that there are 17 of the Top 50 Computers running a version of Windows, ranked by Average Credit on E@H.

Just thought I'd toss in my 2 cents...

George

Proud member of the Old Farts Association

GWGeorge007
GWGeorge007
Joined: 8 Jan 18
Posts: 2767
Credit: 4541011081
RAC: 2153348

Tigers_Dave

Tigers_Dave wrote:

.....Note that I am a professor at a relatively large (~30,000 students) US university.....

Hi Tigers_Dave,

I noticed this little blurb from your post and I congratulate you on funding yourself with the computers that you use at work, essentially a "relatively large (~30,000 students) US university".

I'm a retired Professor from a community college, and I, too, bought out of my own funds many of the items that I used for teaching electronics for nearly 20 years.  It's a small world when you finally get to know a bit more about someone.

Since you don't have a profile set up, if you'd like to communicate further on this topic, feel free to email me a PM.

George

Proud member of the Old Farts Association

Tigers_Dave
Tigers_Dave
Joined: 25 Mar 09
Posts: 228
Credit: 9248626727
RAC: 0

Tom M wrote:Tigers_Dave

Tom M wrote:

Tigers_Dave wrote:

Furthermore, if I started running Windows on my Macs, my IT office would insist on administering those computers.  That is not acceptable to me.  Again, you absolutely have the right to think that philosophy is foolish.

While many do run Windows around here I would be surprised if any Windows machines were current top rankers.  But for standing up a small Farm with modest hardware, if you already had a windows box it would be VERY tempting to drop in a discrete GPU and install Boinc.

Tom M

Tom, I am tempted by your suggestion.  Indeed, I have to purchase a Windows machine to "drive" a digital microscope.  This may lead me to purchase a non-Mac that is connected to the internet, is equipped with a not-too-ancient GPU, crunches E@H tasks, and is administered by my employer's IT office.

"I was born in a small town, and I live in a small town." - John Mellencamp

Tigers_Dave
Tigers_Dave
Joined: 25 Mar 09
Posts: 228
Credit: 9248626727
RAC: 0

mikey wrote: Tom M

mikey wrote:

Tom M wrote:

Hi,

It occurred to me that a thread on setting up multiple slow(er) "farms" like various participants have done might be an interesting topic.

I will note that a "Farm" is a TBD limitation here.  But I will claim that two systems may not qualify. :)

Noisy Tom M 

MANY years ago someone on Seti said a 'farm' was just below a 'ranch' and that it meant more than 10 boxes, I'm not sure that 'box' thing is valid anymore with more and more cpu's out there in the 32 and up range.

That being said I have 17 boxes running Boinc right now and all but 1 has a gpu crunching some project based on my goals and it's capabilities. About half of my boxes are Windows OS's ranging from Win7 to the latest Win11 with the other half of the boxes being various versions of Linux Mint. Unfortunately some projects only run on some OS's and not on others so I'm going to keep the Windows boxes around for a long time is required for my goals. I also have a 'cheese grater' Mac that's now running Linux as it crunches faster and is easier to manage on a day to day basis for me.

I started out on Windows pc's back in the late 70's but didn't start crunching until the late 80's and have never stopped crunching since. MOST of my boxes are 10+ years old from their original purchase date and then I bought them 'off lease' as inexpensive boxes only needing a crunching gpu and a harddrive to be up and running. Buying dual Xeon quad core cpu's that do do HT already installed in a pc case  with 16gb of ram for $139 was a VERY good deal!!!

 

Mikey, seventeen boxes, no matter how cheap, is a tremendous investment in BOINC.  Thank you for this commitment!

"I was born in a small town, and I live in a small town." - John Mellencamp

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.