Troubleshooting Multiple gpu setups that use Riser cards

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6,360
Credit: 9,336,700,609
RAC: 16,062,502

Tom M wrote: a BSOD (Win10)

Tom M wrote:

a BSOD (Win10) error message: "Driver Overran Stack Buffer"

In theory, a Mining Motherboard with 9+ PCIe slots should not have this problem.  My tests have run into some kind of hardware problem related to trying to use Rx 5700's on the latest two Mining MB's. I bought.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Sid
Sid
Joined: 17 Oct 10
Posts: 164
Credit: 961,534,631
RAC: 350,795

Tom M wrote: Tom M wrote: a

Tom M wrote:

Tom M wrote:

a BSOD (Win10) error message: "Driver Overran Stack Buffer"

In theory, a Mining Motherboard with 9+ PCIe slots should not have this problem.  My tests have run into some kind of hardware problem related to trying to use Rx 5700's on the latest two Mining MB's. I bought.

Tom M

I've seen a good advice from miners - you need to use 4 AMD and 4 NVidia cards to make it working in 9+PCIe slots.

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6,360
Credit: 9,336,700,609
RAC: 16,062,502

I have spent the day (and am

I have spent the day (and am not done yet) running my MSI H310-F Pro MB (13 slots) through all the Bios that it has available starting with the bios it shipped with.

I am not certain but I think I have a stronger case for the Bios is screwed up for when I try to run Rx 5000 gpus on it.  I have been playing with 4 (3 Rx 5700's and 1 Rx 5600xt).

So far I have discovered the bios it shipped with doesn't recognize my Radeon gpus at all (V2.20).  However v2.4 does "finally" recognize my gpus.

I may even have gotten 4 of the gpus to run on this MB.  But so far the odd lights come on even when nothing is plugged into a slot issue is still present.

And I managed to find a selection of slots that didn't show up extra lights on the MB but it locks up Linux 20 while booting.

Sometimes I wish I could troubleshoot my electronics/computers with a sledge hammer!

Tom M

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

earthbilly
earthbilly
Joined: 4 Apr 18
Posts: 59
Credit: 1,140,229,967
RAC: 0

I've heard of the issue with

I've heard of the issue with more than 6 slot boards using half Nvidia and half AMD.

It is odd to have this same issue with two different boards at the same time with partial capacity. I only have research to draw from and you are sure the slots you are using are all on the same bus? I have read that one set of slots is connected to one PSU and the second set and if there, the third set wire to different PSU power supply cables. So each set of slots have different power supply busses.

I tried an AMD RX5700XT when they first arrived and had so many problems with it I sent it back to Newegg.

Troubleshoot with ebay instead of a sledgehammer:-)

I guess I have not been any help to you Tom, sorry. I have a fair memory but lack any assembler of being smart.

Work runs fine on Bosons reacted into Fermions,

Sunny regards,

earthbilly

Brandon Clark
Brandon Clark
Joined: 10 Mar 20
Posts: 9
Credit: 248,204,104
RAC: 0

Good discussion, and helpful

Good discussion, and helpful for me. I've been crunching using mainly salvaged hardware for a long time. I was considering building a mining-style rig, but then the thought occurred to me: just take an existing computer with open slots out of their cases and turn them into multiple-GPU rigs.

Are there any beginner pitfalls to watch out for when starting on multiple-GPU systems? My only experience with that so far is an AMD Crossfire two-card system I recently built. I didn't even realize you could install more than one or two cards before a month or two ago.

This system is a mini computer that used to be a mid-grade office machine. It has a single NVIDIA GeForce GT 720 in it. Not very high powered, but those cards are cheap so I was thinking of adding another one for giggles and a little more crunching power.

I also have a Dell 8700 XPS system that has a single Nvidia GeForce GTX 1050 Ti in it currently. It has three PCIe x1 and one PCIe x4 lot still available. This one would need to be taken out of its case and have a bigger power supply added. PCIe risers would also be needed to rebuild this one.

The last system I'm thinking about turning into a mining rig is this beast (Dell Precision desktop). It has an NVIDIA TITAN Xp card in it (talk about an awesome salvage find). It has three open PCIe X16 slots and one PCIe x8. The slots are awfully close together though. I might be able to install one more double-wide card directly on the MB, but anything beyond that would require risers.

All three systems are running Ubuntu.

Lots to think about, and lots to learn. I'm enjoying the forum.

Brandon

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6,360
Credit: 9,336,700,609
RAC: 16,062,502

Brandon Clark wrote: Good

Brandon Clark wrote:

Good discussion, and helpful for me. I've been crunching using mainly salvaged hardware for a long time. I was considering building a mining-style rig, but then the thought occurred to me: just take an existing computer with open slots out of their cases and turn them into multiple-GPU rigs.

Are there any beginner pitfalls to watch out for when starting on multiple-GPU systems?

AFAIK there is no benefit to using a cross-fire gpu linked system with BOINC.

The major issue with multiple gpu BOINC systems is twofold.

Heat and

Exceeding the resource limits for PCIe.

Heat can be managed by taking the cover off the PC box and directing a fan in there.   You can also go to the mining chassis or to liquid cooling for your GPUs.

The Resource limits of PCIe are usually something that only happens when you get up past 3 or 4 GPUs.  If your bios has a use "Above 4G" setting (usually near the PCIe stuff in the bios) you can often run over 4 GPUs without a strain.

You can often get up to 5 GPUs on a system.  But it becomes problematic at 6 or above.

The Riser card kits often need a replacement cable between the PCIe to usb3 adapter and the riser card base.

If you can find a gtx 1060 3gb for cheap enough it will run at least one task at a time on a lot of BOINC projects.  It will NOT run the Gravity Wave apps at E@H. It doesn't have enough memory.  It will run a Gamma Ray task.

A Gtx 750 or 750ti will process faster than a gt710.

An Rx 470/480 (4 GB or 8 GB) should run Gamma-Ray tasks pretty fast.  The 8 GB card should run at least 2 GR at the same time.  Certainly, the Rx 570/580's do.

Tom M

 

 

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

Brandon Clark
Brandon Clark
Joined: 10 Mar 20
Posts: 9
Credit: 248,204,104
RAC: 0

Thanks for the tips Tom. I've

Thanks for the tips Tom. I've never checked in the BIOS for that setting before, but now that you mention it I do remember reading about something similar before. I'll check each system to see what the BIOS has available.

That's also good to know about the memory needs for these projects, and the card recommendations. Sifting through available GPUs online can be maddening.

A quick follow-up question: it seems like most high end graphics cards are using external power going directly from the PSU to the card (mine are all set up that way). When using a riser card does the riser have to carry much power, if the graphics card has power direct from the PSU?

The reason I ask is that the PCI slots I'm considering expanding into are rated for considerably lower wattage than the main PCI slots that are intended for graphics cards.

Thanks

Tom M
Tom M
Joined: 2 Feb 06
Posts: 6,360
Credit: 9,336,700,609
RAC: 16,062,502

Brandon Clark wrote: The

Brandon Clark wrote:

The reason I ask is that the PCI slots I'm considering expanding into are rated for considerably lower wattage than the main PCI slots that are intended for graphics cards.

The mining kits that I am using draw power directly from the PSU, not through the Motherboard.

If you use flat ribbon-based kits then the base power (AFAIK) is drawn through the MB PCIe slots.

Most of the time the standard Mining riser card kits which don't use the flat-ribbon are perfectly fine.

Ian&Steve and Keith? participate in the rare BOINC project that is sensitive to the bandwidth between the CPU and gpu.  E@H is not particularly (so far) sensitive to the bandwidth.

Tom M

 

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)  I want some more patience. RIGHT NOW!

archae86
archae86
Joined: 6 Dec 05
Posts: 3,157
Credit: 7,201,034,931
RAC: 908,169

Brandon Clark wrote:A quick

Brandon Clark wrote:
A quick follow-up question: it seems like most high end graphics cards are using external power going directly from the PSU to the card (mine are all set up that way).

The cards themselves pull power both from the PCI-e connector and from the top side PSU plugs.  Neither is optional.  So the question is how your particular arrangement obtains the power it delivers to the PCI-e connector.

Brandon Clark
Brandon Clark
Joined: 10 Mar 20
Posts: 9
Credit: 248,204,104
RAC: 0

That makes me wonder if power

That makes me wonder if power supply problems are a source for a lot of the instability we hear about with risers. Just because we can install a one-to-four PCIe splitter on a motherboard slot it doesn't mean that particular slot has the watts available to meet the needs of that many cards . . . . One one motherboard I have the slots intended for graphics cards are rated at 225 watts, while the ones for general usage are only rated at 25 watts.

Another riser question: With one of my systems the board has two PCIe x16 slots that are already intended for graphics cards. I want to go to risers for better cooling though. Would the ribbon style risers that take up the full x16 length be the better option for those kinds of slots, since all I am doing is relocating a single card?

Then, to add other cards I could use the USB style risers that only take up an x1 slot?

Thanks

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.