Troubleshooting Multiple gpu setups that use Riser cards

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

Tom M wrote: Apparently, I

Tom M wrote:

Apparently, I ordered the 90 degrees long ones rather than the straight-through version. I have managed to get them installed anyway.  I will try ordering a couple more of the shorter straight through and a long one again later.

The system has booted showing all four flat ribbon cables working.  But until the day cools down I can't run it anymore.

Tom M

I don't know if I should laugh or cry.  The flat ribbon cables may have knocked a minute off the average processing time of the Gamma Ray #1 gpu tasks.  (single thread time).

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Keith Myers
Keith Myers
Joined: 11 Feb 11
Posts: 4701
Credit: 17545359233
RAC: 6412061

Compared to what

Compared to what previously?

I'm guessing whatever you were running before caused lots of retransmits on the PCIe bus because of transmission errors.

And the flat ribbon cables are doing a much better job of preserving signal integrity and therefore has much fewer retransmits and so data flows more efficiently between cpu, memory and the gpu.

So faster crunching of tasks.

 

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

It looks like my XFX Rx 5700

It looks like my XFX Rx 5700 has "bit the dust".  Neither the Ubuntu BOINC nor the Windows 10 BOINC recognizes it.  And Windows 10 doesn't recognize it in Device Manager.

Yet it will still drive a video monitor.  It has a broken off tab to the Bios switch which has made it tough to toggle that.  I opened a support ticket.

Another Rx 5700 reference card is failing after extended processing (less than 24 hours though).  I tried switching flat ribbon cables.  That didn't help.  I am trying all cards on full length slots.  Next I will try a different full length slot.

"I would still like to run 5 gpus," he said wishfully.

Well at lest I got a Tom Swifty out of it.

:(

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

Got in another (single ribbon

Got in another (single ribbon instead of multiple thinner ribbons) and at first, I thought it was DOA which seems to be unlikely for a new/still in its packaging.

Before I got done testing it appears I have 3 "topline" PCIe cables (all 8pin + 6pin) that are malfunctioning.

So I now am testing 3 GPUs on flat ribbon cables for the base inputs.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

Ian&Steve C. wrote: just get

Ian&Steve C. wrote:

just get a reasonable quality riser cable. do not get those generic grey un-shielded cables. get the ones that are PCIe gen 3 capable. usually in a thicker black coating. I use these ones with relatively good success, but it looks like they aren't available at the moment, but others like it exist just search around on amazon. for most of my cards I'm using the 20cm length cable on my mining frames (the furthest away use 30cm), but you should figure out what length you need specifically if you're not using a mining frame. also keep in mind that you might need a right angle cable or a right angle adapter for the 2 GPUs that will sit over the CPU area if you're using a tall-ish CPU heatsink.

edit: looks like the same brand riser is available under a new part number. I guess they updated it: https://www.amazon.com/EZDIY-FAB-Express-Flexible-Extension-Upgrade/dp/B07MDLYBJ4/ref=sr_1_5?dchild=1&keywords=pcie+3.0+riser+cable+20cm&qid=1622216502&s=electronics&sr=1-5

Ian&SteveC.

It looks like those are completely out of stock wherever I have googled.

Do you have any alternative suggestions?

Thank you.

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3681
Credit: 33832023313
RAC: 37722704

Tom M wrote:Ian&Steve C.

Tom M wrote:

Ian&Steve C. wrote:

just get a reasonable quality riser cable. do not get those generic grey un-shielded cables. get the ones that are PCIe gen 3 capable. usually in a thicker black coating. I use these ones with relatively good success, but it looks like they aren't available at the moment, but others like it exist just search around on amazon. for most of my cards I'm using the 20cm length cable on my mining frames (the furthest away use 30cm), but you should figure out what length you need specifically if you're not using a mining frame. also keep in mind that you might need a right angle cable or a right angle adapter for the 2 GPUs that will sit over the CPU area if you're using a tall-ish CPU heatsink.

edit: looks like the same brand riser is available under a new part number. I guess they updated it: https://www.amazon.com/EZDIY-FAB-Express-Flexible-Extension-Upgrade/dp/B07MDLYBJ4/ref=sr_1_5?dchild=1&keywords=pcie+3.0+riser+cable+20cm&qid=1622216502&s=electronics&sr=1-5

Ian&SteveC.

It looks like those are completely out of stock wherever I have googled.

Do you have any alternative suggestions?

Thank you.

Tom M

just type "pcie 3.0 riser cable 20cm" into the search field on Amazon and see what comes up. there are several options. available. you don't have to use exactly the part I used, they're all pretty much the same product.

 

like this: https://www.amazon.com/Express3-0-Extension-Gold-Plated-Connector-Degree/dp/B07RDFF2DP/ref=sr_1_4?dchild=1&keywords=pcie+3.0+riser+cable+20cm&qid=1625507893&s=electronics&sr=1-4

 

just pay attention to if it's a straight riser, or a right angle riser, what direction the right angle is (right/left, 90/270, etc) and try searching for different lengths if 20cm doesn't work for you.

_________________________________________________________________________

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

TY

TY

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

Ian&Steve C. wrote: just

Ian&Steve C. wrote:

just type "pcie 3.0 riser cable 20cm" into the search field on Amazon and see what comes up. there are several options. available. you don't have to use exactly the part I used, they're all pretty much the same product.

 

like this: https://www.amazon.com/Express3-0-Extension-Gold-Plated-Connector-Degree/dp/B07RDFF2DP/ref=sr_1_4?dchild=1&keywords=pcie+3.0+riser+cable+20cm&qid=1625507893&s=electronics&sr=1-4

 

just pay attention to if it's a straight riser or a right angle riser, what direction the right angle is (right/left, 90/270, etc) and try searching for different lengths if 20cm doesn't work for you.

Just ordered two from the Amazon vendor and three from an eBay vendor in the USA.  All are 180 degrees (straight through).

Tom M

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Tom M
Tom M
Joined: 2 Feb 06
Posts: 5586
Credit: 7674209461
RAC: 1795069

I had an EPYC running 4 rx

I had an EPYC running 4 rx 5700's reliably.  But I also had a "spare" rx 5700.

I am now down to 0 gpus that process through flat ribbon cables.  I had 3 rx 5700's that were working.  But they now are all stalling/not using cpu cycles after X time.

Will start by adding one gpu next.

Tom M

 

A Proud member of the O.F.A.  (Old Farts Association).  Be well, do good work, and keep in touch.® (Garrison Keillor)

Ian&Steve C.
Ian&Steve C.
Joined: 19 Jan 20
Posts: 3681
Credit: 33832023313
RAC: 37722704

Tom M wrote:I had an EPYC

Tom M wrote:

I had an EPYC running 4 rx 5700's reliably.  But I also had a "spare" rx 5700.

I am now down to 0 gpus that process through flat ribbon cables.  I had 3 rx 5700's that were working.  But they now are all stalling/not using cpu cycles after X time.

Will start by adding one gpu next.

Tom M

 

did you make any software changes between 4x GPUs working and now nothing working?

 

try 1 GPU on 1 riser.

if it still gives issues. change the riser (only) to another one.

if it gives issues with all risers, try plugging the single GPU directly into the motherboard.

if it still gives issues, try a different slot.

if it still gives issues, try a different GPU, and repeat.

if it still gives issues, try a different PSU or different power connections.

 

changing one thing at a time can help you isolate where the problem is.

 

_________________________________________________________________________

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.