A quick guide: How to install OpenCL for AMD GPUs on Linux Kubuntu 18.04 (and similar distro)

mikey
mikey
Joined: 22 Jan 05
Posts: 11888
Credit: 1828065059
RAC: 206393

robl wrote:Today I noted that

robl wrote:

Today I noted that an Ubuntu PC was offering to upgrade to the newest release so I opted to do this.  Got all the way through the process and encountered total failure on the reboot.  Could not get passed the Gnome Display manager.  No matter what I did I was stuck so I loaded up a USB thumb drive and reloaded a new release of Ubuntu.  I then "rejoined" einstein@home but could not get any gpu work.  I forgot that I had a Radeon RX 550 card.  This card would need AMD driver support so here is what I did in a cut and paste format:

  1. download amd drivers for card
  2. to find out what card you have on ubuntu: lspci | grep VGA
  3. 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev cf)
  4. go to amd website and download the drivers for the Radeon RX 550.
  5. place the driver file ending in xz in the $HOME/Download directory 
  6. unwindit it: tar -xf amdgpu-pro-19.20-812932-ubuntu-18.04.tar.xz
  7. run the following script in the "xz" file:
  8. ./amdgpu-pro-install -y --opencl=pal,legacy
  9. reboot the pc when the script finishes. you should now be ready to
  10. crunch on einstein@home i.e. download gpu Work Units.

I will be trying this this coming week on one of my own pc's, I'm not a Linux person so am hoping you didn't leave out any steps or I will a very lost puppy. I have a few pc's that do not like Win10 so am exploring my options from WinXP to Linux. I can get the Nvidia cards to work in Linux but have never been successful with the AMD cards since they got rid of the built in drivers. Step by step is perfect for me, thank you very much!!

robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454480658
RAC: 8819

mikey wrote:robl wrote:Today

mikey wrote:
robl wrote:

Today I noted that an Ubuntu PC was offering to upgrade to the newest release so I opted to do this.  Got all the way through the process and encountered total failure on the reboot.  Could not get passed the Gnome Display manager.  No matter what I did I was stuck so I loaded up a USB thumb drive and reloaded a new release of Ubuntu.  I then "rejoined" einstein@home but could not get any gpu work.  I forgot that I had a Radeon RX 550 card.  This card would need AMD driver support so here is what I did in a cut and paste format:

  1. download amd drivers for card
  2. to find out what card you have on ubuntu: lspci | grep VGA
  3. 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev cf)
  4. go to amd website and download the drivers for the Radeon RX 550.
  5. place the driver file ending in xz in the $HOME/Download directory 
  6. unwindit it: tar -xf amdgpu-pro-19.20-812932-ubuntu-18.04.tar.xz  
  7. This will generate a directory that contains lots of files.  
  8. cd to the above directory and run in that directory step 9
  9. ./amdgpu-pro-install -y --opencl=pal,legacy
  10. reboot the pc when the script finishes. 
  11. you should now be ready to crunch on einstein@home i.e. download gpu Work Units.

I will be trying this this coming week on one of my own pc's, I'm not a Linux person so am hoping you didn't leave out any steps or I will a very lost puppy. I have a few pc's that do not like Win10 so am exploring my options from WinXP to Linux. I can get the Nvidia cards to work in Linux but have never been successful with the AMD cards since they got rid of the built in drivers. Step by step is perfect for me, thank you very much!!

I modified steps 7 and 8 above for clarity.  

mikey
mikey
Joined: 22 Jan 05
Posts: 11888
Credit: 1828065059
RAC: 206393

robl wrote:mikey wrote:robl

robl wrote:
mikey wrote:
robl wrote:

Today I noted that an Ubuntu PC was offering to upgrade to the newest release so I opted to do this.  Got all the way through the process and encountered total failure on the reboot.  Could not get passed the Gnome Display manager.  No matter what I did I was stuck so I loaded up a USB thumb drive and reloaded a new release of Ubuntu.  I then "rejoined" einstein@home but could not get any gpu work.  I forgot that I had a Radeon RX 550 card.  This card would need AMD driver support so here is what I did in a cut and paste format:

  1. download amd drivers for card
  2. to find out what card you have on ubuntu: lspci | grep VGA
  3. 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 550 640SP / RX 560/560X] (rev cf)
  4. go to amd website and download the drivers for the Radeon RX 550.
  5. place the driver file ending in xz in the $HOME/Download directory 
  6. unwindit it: tar -xf amdgpu-pro-19.20-812932-ubuntu-18.04.tar.xz  
  7. This will generate a directory that contains lots of files.  
  8. cd to the above directory and run in that directory step 9
  9. ./amdgpu-pro-install -y --opencl=pal,legacy
  10. reboot the pc when the script finishes. 
  11. you should now be ready to crunch on einstein@home i.e. download gpu Work Units.

I will be trying this this coming week on one of my own pc's, I'm not a Linux person so am hoping you didn't leave out any steps or I will a very lost puppy. I have a few pc's that do not like Win10 so am exploring my options from WinXP to Linux. I can get the Nvidia cards to work in Linux but have never been successful with the AMD cards since they got rid of the built in drivers. Step by step is perfect for me, thank you very much!!

I modified steps 7 and 8 above for clarity.  

Thank you very much!!

cecht
cecht
Joined: 7 Mar 18
Posts: 1421
Credit: 2444685658
RAC: 1502355

The new Linux kernel 5.0.0

The new Linux kernel 5.0.0 has AMDGPU drivers built-in: from kernel.org, "The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core Next (GCN) architecture.", which includes Polaris and Vega architectures. It doesn't include OpenCL needed for E@H crunching, so that needs to be installed from the downloaded AMDGPU driver stack, as others have described.  AMDGPU drivers have been part of the kernel since 4.15, which is why Millenium began this forum, way back when, describing how to install OpenCL on top of built-in drivers. There has since been lots of discussion about matching OS versions, kernel versions, AMDGPU driver versions, and installing OpenCL with various combination thereof.

What I am still hazy on is whether there is any performance difference for E@H crunching between the AMDGPU Pro stack and its All-Open stack. This from the AMDGPU documentation:
===================
Pro: recommended for use with Radeon Pro graphics products.
All-Open: recommended for use with consumer products.

All-Open Stack     
    Base kernel drivers
    Base accelerated graphics drivers
    Mesa OpenGL
    Mesa multimedia

Pro Stack     
    Base kernel drivers
    Base accelerated graphics drivers
    Mesa multimedia
    Pro OpenGL
    Pro OpenCL
            PAL OpenCL stack (supports Vega 10 and later products)
            Legacy OpenCL stack (supports legacy products older than Vega 10)
    Pro Vulkan
===================

I'm guessing that the latest Linux kernel has the equivalent of the All-Open driver stack.(?)
So, given Linux kernel 5.0 and not having Vega 10 or Vega 20 GPUs, would there be any crunching performance advantage to installing just OpenCL(legacy) over the system's default drivers vs. installing the full Pro stack with OpenCL(legacy)?

Ideas are not fixed, nor should they be; we live in model-dependent reality.

robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454480658
RAC: 8819

cecht wrote:The new Linux

cecht wrote:

The new Linux kernel 5.0.0 has AMDGPU drivers built-in: from kernel.org, "The drm/amdgpu driver supports all AMD Radeon GPUs based on the Graphics Core Next (GCN) architecture.", which includes Polaris and Vega architectures. It doesn't include OpenCL needed for E@H crunching, so that needs to be installed from the downloaded AMDGPU driver stack, as others have described.  AMDGPU drivers have been part of the kernel since 4.15, which is why Millenium began this forum, way back when, describing how to install OpenCL on top of built-in drivers. There has since been lots of discussion about matching OS versions, kernel versions, AMDGPU driver versions, and installing OpenCL with various combination thereof.

What I am still hazy on is whether there is any performance difference for E@H crunching between the AMDGPU Pro stack and its All-Open stack. This from the AMDGPU documentation:
===================
Pro: recommended for use with Radeon Pro graphics products.
All-Open: recommended for use with consumer products.

All-Open Stack     
    Base kernel drivers
    Base accelerated graphics drivers
    Mesa OpenGL
    Mesa multimedia

Pro Stack     
    Base kernel drivers
    Base accelerated graphics drivers
    Mesa multimedia
    Pro OpenGL
    Pro OpenCL
            PAL OpenCL stack (supports Vega 10 and later products)
            Legacy OpenCL stack (supports legacy products older than Vega 10)
    Pro Vulkan
===================

I'm guessing that the latest Linux kernel has the equivalent of the All-Open driver stack.(?)
So, given Linux kernel 5.0 and not having Vega 10 or Vega 20 GPUs, would there be any crunching performance advantage to installing just OpenCL(legacy) over the system's default drivers vs. installing the full Pro stack with OpenCL(legacy)?

Like you said drivers alone are not enough.  you need OpenCl support.  That is where the command

"./amdgpu-pro-install -y --opencl=pal,legacy" comes in.  I downloaded the full driver tar to get this script and other supporting files to install OpenCl support required by E@H.  Before i ran this script I could not get any E@H AMD GPU WUs even though I had the latest Ubuntu release with Kernel 5.  

cecht
cecht
Joined: 7 Mar 18
Posts: 1421
Credit: 2444685658
RAC: 1502355

robl wrote:Like you said

robl wrote:

Like you said drivers alone are not enough.  you need OpenCl support.  That is where the command

"./amdgpu-pro-install -y --opencl=pal,legacy" comes in.  I downloaded the full driver tar to get this script and other supporting files to install OpenCl support required by E@H.  Before i ran this script I could not get any E@H AMD GPU WUs even though I had the latest Ubuntu release with Kernel 5.

And there is the method offered by Koschi offered earlier is this forum to build a DEB package for just OpenCL that can be used with any(?) recent Linux distribution. (I'm not sure how much the OpenCL version matters.)

A few weeks back, Gary Roberts started this discussion forum and in his initial post mentions that he too wrote a script to extract and load OpenCL libraries for various hosts.

So given that OpenCL can be loaded independently of AMDGPU stack installation, I was just wondering whether there is any crunching speed advantage of amdgpu-pro drivers over the drivers built-in to recent Linux distributions. If I (have to) do a system reinstall, I'll test that and post the results.

Ideas are not fixed, nor should they be; we live in model-dependent reality.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109385746815
RAC: 35923947

cecht wrote:... I was just

cecht wrote:
... I was just wondering whether there is any crunching speed advantage of amdgpu-pro drivers over the drivers built-in to recent Linux distributions. If I (have to) do a system reinstall, I'll test that and post the results.

The bit that is "built-in" (and it's the same for all Linux distros) is the amdgpu kernel module.  If you are using a particular kernel version, you should use the particular version of amdgpu (and all other kernel modules for that matter) built at the same time from the full kernel source tree.  The amdgpu kernel module is an open source contribution by AMD to the Linux kernel project and is part of the kernel source tree.  It is the basis on top of which you need to install further library components which supply OpenCL.

The OpenCL libraries are not "built-in".  There is a mixture of all sorts of bits and pieces, both open source and proprietary, in the "Radeon Software for Linux" - formerly officially known and currently unofficially still known as AMDGPU-PRO.  The actual OpenCL libs are just a small subset of the total of about 50 different packages that make up the full set.  There are around 5 or 6 packages only that are needed for OpenCL.  I had bought a Polaris GPU (RX 460) nearly 3 years ago and needed the OpenCL components from the 16.60 version of that package so was forced to work out what parts were needed.

I did that by carefully studying the amdgpu-pro-install script at the time.  If I remember correctly, there was a --compute option that pointed me in the right direction.  That option was later deprecated and replaced with --headless --opencl=....  I can remember reading a note about --headless being the way to NOT replace the already included OpenGL stuff from the original OS install with whatever other packages from -PRO that could also provide some different version of OpenGL.  Since the graphics side of things was working fine and I just wanted to add OpenCL, I felt that was probably the best policy.  It works absolutely fine for my very-much-non-supported distro situation :-).  For those with a supported distro, just do something along the lines that robl suggested, and perhaps add the --headless option to what he showed.  In fact, that's exactly what the OP of this thread posted.

It is permissible to have multiple OpenCL implementations installed.  For that reason, I always selected the packages that were pointed to by the --opencl=legacy,pal variant of that option.  My understanding is that the 'legacy' setting is used for GPUs prior to Vega (eg Polaris) whereas 'pal' is for vega or later.  I decided that by having both, it would be possible to just replace a Polaris GPU with Vega or later and have it work out of the box.  As yet, I don't own anything later than Polaris so haven't tested that.

Now to get to the actual 'meat' of your quote.  I have used many versions of the OpenCL components from AMDGPU-PRO from 16.60 to present.  The versions I have run are 16.60, 17.40, 17.50, 18.10, 18.20, 18.30, 18.50, and 19.10.  The short answer is that I have experienced very little difference in compute speed/efficiency/whatever between any of them.  I should hasten to add that the testing methodology used was extremely inadequate - on any reasonable standard.  So, if you want to do it more rigorously, please be our collective guest :-).

When I was testing for benefits of newer versions, I would select a test host or two and upgrade the full machine to the latest of everything (including the kernel and amdgpu module).  I would then use my script to apply the new version of the OpenCL bits from AMDGPU-PRO.  I would then start BOINC and make sure the GPU was still being detected.  I would take a pretty rough average of a page of crunch times for the previous version and compare that with the average for the next page of crunch times for the new version, making sure to avoid any possible 'gap' tasks.  The numbers always turned out to be quite similar so I wouldn't bother to upgrade any further hosts.  It continued that way until 18.30.  At that point most of the Polaris fleet was still on 16.60 and the test machines plus new installs were on 18.30.

The object of all this was NOT to test for better OpenCL performance, although that would have been a nice byproduct.  I was dealing with a different issue completely.   In late 2017 and on into 2018, I was having an issue with RX 460 GPUs stalling/locking up extremely regularly around the 25 day uptime mark.  If you're interested, I discussed the issue in some detail in this message and in a subsequent message, two messages later.  In 2018, the same thing was happening with RX 570/580 GPUs except it was on a much shorter timescale - around 12-15 days uptime.  I wasn't sure what was causing the issue but I suspected the amdgpu kernel module or perhaps the OpenCL bits.

For that reason, I was very keen to both upgrade the kernel+module as well as the OpenCL bits on test machines whenever there were new versions.  It was a tedious process because I wouldn't know if there had been a resolution of the problem until either the 15 day or 25 day 'limit' had been well and truly exceeded.  At the time I first became confident enough that there were longer uptimes and they were for real, the 18.X0 versions of AMDGPU-PRO were the current ones and when I had uptimes getting beyond the 60+ days mark, 18.30 was available.  I was just elated that somebody had finally fixed whatever it was that was causing the problem.  I regarded the task of upgrading every single host to the magical kernel+amdgpu+OpenCL combination as a true pleasure rather than as a chore :-).  I suspect it was the amdgpu module rather than OpenCL but I didn't actually spend time trying to prove one way or the other.

As a result, everything got upgraded to 18.30 and there was no urgency to go much further than that.  I have incorporated both 18.50 and 19.10 into my script as well.  Initially I had trouble getting those to work but solved that by discovering that I needed to add an extra component to the LD_LIBRARY_PATH environment variable.  I have casually run both of those versions on test machines and nothing much seems to have changed.  If someone happened to come up with some evidence that crunch times were improved for either 19.20 or 19.30 (or whatever comes next) I could probably get motivated to do a mass upgrade once again :-).

Cheers,
Gary.

cecht
cecht
Joined: 7 Mar 18
Posts: 1421
Credit: 2444685658
RAC: 1502355

Gary Roberts wrote:It is

Gary Roberts wrote:

It is permissible to have multiple OpenCL implementations installed.  For that reason, I always selected the packages that were pointed to by the --opencl=legacy,pal variant of that option.  My understanding is that the 'legacy' setting is used for GPUs prior to Vega (eg Polaris) whereas 'pal' is for vega or later.  I decided that by having both, it would be possible to just replace a Polaris GPU with Vega or later and have it work out of the box.  As yet, I don't own anything later than Polaris so haven't tested that.

Now to get to the actual 'meat' of your quote.  I have used many versions of the OpenCL components from AMDGPU-PRO from 16.60 to present.  The versions I have run are 16.60, 17.40, 17.50, 18.10, 18.20, 18.30, 18.50, and 19.10.  The short answer is that I have experienced very little difference in compute speed/efficiency/whatever between any of them.  I should hasten to add that the testing methodology used was extremely inadequate - on any reasonable standard.  So, if you want to do it more rigorously, please be our collective guest :-).

Thank you very much for that info and for all the other juicy details you provided. I'm now working on trying to get Koschi's script to run and have some ideas to approach it differently, so that should keep me busy for a while. Not that there is anything wrong with using amdgpu's install script for OpenCL, I just want to have a play with scripting. :)

Ideas are not fixed, nor should they be; we live in model-dependent reality.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109385746815
RAC: 35923947

cecht wrote:... I'm now

cecht wrote:
... I'm now working on trying to get Koschi's script to run and have some ideas to approach it differently, so that should keep me busy for a while. Not that there is anything wrong with using amdgpu's install script for OpenCL, I just want to have a play with scripting. :)

I'm very pleased to hear that!  The ability to do some basic scripting is a very useful skill to possess for someone who wants to use Linux to its full potential.

I looked at what Koschi was doing when he first mentioned it.  I haven't looked more recently so I'm not sure if it has changed in any significant way from what I saw at the time.  These things can quickly evolve over fairly short time scales :-).  The script is tailored to Ubuntu style distros (.deb packaging) and does things a bit differently to the way I approached the problem.

I do understand what he was doing and why so if there is anything that puzzles you, please ask and I'll try to explain.  You have a few hurdles to jump - it's not just a matter of understanding the syntax of the shell (bash).  You have to come to grips with some pretty powerful (and arcane) traditional Unix utilities like sed and awk, etc.

I'm quite keen to follow your progress so do remember to keep us all informed from time to time :-).

Cheers,
Gary.

cecht
cecht
Joined: 7 Mar 18
Posts: 1421
Credit: 2444685658
RAC: 1502355

koschi wrote:...I focused

koschi wrote:

...I focused back in the AMDGPU-PRO driver and looked for the libraries that it needs to get OpenCL running on unsupported distributions and kernels. Arch Linux had a build script to extract the libraries from the official driver, I based my version for DEB based distributions around that:

http://kerbodyne.com/boinc/create_amdgpu-ocl-pkg.sh

Place the script with the driver archive of 18.50 in the same directory (I'm still working on automatic downloading from the official website), then execute the script. It unpacks the driver and extracts libraries from 3 different packages, then creates an installable package from these. So they can be installed/removed quickly, copied to other systems etc. I choose to provide a script here, so I don't (potentially) violate the license agreement by redistributing AMDs software on my server. The script will ask for sudo permissions in order to change the ownership of the files contained in the archive to root:root. Have a look at the code to confirm that it doesn't do any harm. I created it in an isolated environment, also tried it on my primary workstation, no harm was done running the script ;-)

I could get a Polaris and a VEGA card crunching on Ubuntu 19.04 with kernel 5.0.0 via this procedure. No success yet on pre-Polaris cards yet, I plan trying with an HD7950 tonight though, lets see.

Any feedback is welcome, I'd appreciate you reporting back in case it works or fails for you.

Thanks!

Once I figured out to include the archive name as a command line argument, it worked great! To help future numbskulls who may have downloaded the script but lost track of your instructions here, you may want to add something like this between lines 10 and 12:

scriptversion=0.3

#Set up basic use instructions
usage() {
    cat <<END_USAGE
 USAGE: $0 amdgpu-pro-<version>.tar.xz
    ...OR...
 USAGE: $0 amdgpu-pro-<version>

    This script will extract several OpenCL libraries from the official
      amdgpu-pro drivers.
    They are then packaged into a DEB file that can be easily installed or
      removed through dpkg.
    The last line of the script output will say how to install the DEB package.
    There is no need to install any other components to get OpenCL support.
    Place this script in the same directory with the amdgpu-pro driver tar.xz
      archive (downloaded from https://www.amd.com/support)
      ...OR with the previously extracted archive directory; either can be used
      for the command line execution.
    Once executed, you may be prompted to provide sudo authorization.
    When completed, the working directory will contain the amdgpu-ocl DEB file
      and the extracted AMDGPU-PRO drivers directory, which you can delete
      or save for other installations.

 >>>Please don't work in / (root), as this script might destroy your system.  
 
END_USAGE
}
if [[ $# -ne 1 || $1 == "--help" || $1 == "-h" ]]; then
    usage
    exit 1
fi

#archivename='amdgpu-pro-19.20-812932-ubuntu-18.04.tar.xz'
archivename=$1


To allow an option to use a previously extracted AMDGPU-PRO archive, which happens pretty much automatically on my system when I download archives, you can add an IF condition to the archiveprocessing() function at line 42:



        if [[ ${driverarchivename} =~ ".tar.xz" ]]; then
            echo "Processing driver archive:"
            printf " Extracting ${driverarchivename} "
            tar xJf ${driverarchivename} && printf " done\n"
        fi


If a user enters the extracted driver dir name as an argument instead of the tar.xz archive name, then the script will  proceed past this archive extraction step and on to the DEB package extractions. 

Hope this helps.

EDIT: Made the usage() function prettier with a  here document

Ideas are not fixed, nor should they be; we live in model-dependent reality.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.