Impossible GPU tasks (floating point) received
13 Sep 2018 13:19:38 UTC
Topic 216337
(moderation:
My workunit has a Radeon HD 5770 GPU which get detected as:
AMD AMD JUNIPER (DRM 2.50.0 / 4.15.0-33-generic, LLVM 6.0.0) (1024MB)
This GPU how-ever does not support double presision floating point operations, causing the computation to fail:
21:59:46 (1499): [debug]: Flags: X64 SSE SSE2 GNUC X86 GNUX86
21:59:46 (1499): [debug]: glibc version/release: 2.27/stable
21:59:46 (1499): [debug]: Set up communication with graphics process.
boinc_get_opencl_ids returned [0x25ef148 , 0x7f5c854b2120]
Using OpenCL platform provided by: Mesa
Using OpenCL device "AMD JUNIPER (DRM 2.50.0 / 4.15.0-33-generic, LLVM 6.0.0)" by: AMD
Max allocation limit: 751619276
Global mem size: 1073741824
OpenCL compiling FAILED! : -11 . Error message: input.cl:7:26: error: unsupported OpenCL extension 'cl_khr_fp64' - ignoring
input.cl:10:30: error: unknown type name 'double2'; did you mean 'double'?
input.cl:10:30: error: use of type 'double' requires cl_khr_fp64 extension to be enabled
.
OpenCL device has no FP64 support
How-ever I keep getting GPU tasks assigned, unless I explicitly disable them in the web configuration.
I be happy to write patches, but have no glue where to get started, is this a platform (BOINC) problem or should it be captured by individual projects?
I may be wrong (and have been
)
I may be wrong (and have been many times before!) but I suspect your GPU is too old for use here, according to AMD the HD5770 supports Open CL 1.0 and the minimum requirement for current GPU tasks at Einstein is for cards supporting Open CL 1.2 and above.
rickvanderzwet wrote:... This
)
Hi Rick,
Welcome to Einstein@Home. I'm sorry you've had some setbacks whilst getting started.
I don't think the problem is the lack of FP64. My gut feeling is that the real issue is that the driver you are using doesn't expose the full OpenCL capabilities of your card. According to this entry on Wikipedia, your card does support OpenCL 1.2. I believe that capability is not being properly recognised.
A task is processed in two stages. The initial stage (which does about ~90% of the work) only uses single precision. During this stage, potential candidate signals are identified. There is a follow-up stage (the final ~10%) which does need DP. This 2nd stage re-evaluates the 'toplist', the 10 most likely candidates. If the GPU doesn't have any DP capability, the follow-up stage is supposed to be performed using a CPU core. Of course, this will slow down that last part of the calculations, but it's not supposed to prevent a task from starting correctly in the first place.
I don't have anything as old as a 5770 but I do have a 7770 and a number of 7850s. I run all mine using PCLinuxOS. I've never used Ubuntu so have no experience there. I maintain older versions of PCLOS in order to use the deprecated fglrx driver. I suspect that if you were able to run fglrx on an older version of Ubuntu, your card may be usable. I don't know at what point fglrx was deprecated on Ubuntu.
At the end of the day, I think your card could work but the performance won't be great. The 7770 was the first AMD GPU I used at Einstein. I continue to run it because it's still quite productive. In the pre-GPU days, I had a lot of CPU only crunchers (2008/2010 vintage). They were due for retirement quite a while ago. I stuck an RX 460 (or better) in a lot of them and cut out most of the CPU crunching. I think it's a good way to rejuvenate a machine that's too slow for CPU tasks but still running fine. GPU crunch times are largely independent of the speed of the CPU.
Cheers,
Gary.
Gary Roberts wrote: I don't
)
Thanks, spot on!
14.04 which is still supported as LTS. I will give it a shot.
Performance will indead not be great, the system how-ever is used as alternative to electric heating, and allows re-use of the hardware instead of disposal which is better in the terms of waste hierarchy. Helping a interesting project is nice bonus :-).
After quite some tinkering,
)
After quite some tinkering, got the proper fglrx driver installed under Ubuntu 14.04.5 amd64 LTS
Brief install guide
https://www.amd.com/en/support/graphics/amd-radeon-hd/ati-radeon-hd-5000-series/ati-radeon-hd-5770$ sudo apt-get dist-upgrade$ sudo apt-get install boinc-amd-opencl$ ar p fglrx_15.201-0ubuntu1_i386_UB_14.01.deb data.tar.gz | sudo tar -C / -xzf - ./etc/ati/amdpcsdb.default$ (cd /tmp; ln -s libOpenCL.so.1 libOpenCL.so)The irony
All hard work in trying to get the GPU to work has proven to be useless. The scheduler log shows no work assigned to my Radeon HD 5770 GPU since mine only has 512MB memory available and at least 800MB is required :-)
The conclusion
I need to find myself more recent GPU, with at least 1GB onboard memory or find GPU tasks with smaller memory requirements.
rickvanderzwet wrote:... I
)
I'm confused!
In the second line of your opening post, the GPU was detected as having 1GB (1024MB). That should certainly be sufficient. I run a HD7770 with that amount of VRAM with no problem and the machine is running a full KDE desktop.
I imagine there is something else to tweak to stop the supplied memory figure being set at only 536870912, which is sufficiently more than 512MB to indicate you do indeed already have a 1GB card.
Cheers,
Gary.
Gary Roberts
)
You are right (as usual). The card comes in two variants 512MB and 1GB. After visual inspection the marking on my card clearly shows 1GB, so something is clearly wrong here.
Calculation matched up 536870912 bytes / 1024 / 1024 = 512MB, which tricked me in thinking this was correct. The search will continue...
It turns out to be setting a
)
It turns out to be setting a hidden environment variable which allows exposing the memory.
$ clinfo | grep -e 'Max memory' -e 'Global memory size' | head -2 Max memory allocation: 134217728 Global memory size: 536870912$ GPU_MAX_ALLOC_PERCENT=95 GPU_MAX_HEAP_SIZE=95 clinfo | grep -e 'Max memory' -e 'Global memory size' | head -2 Max memory allocation: 254803968 Global memory size: 1019215872In order to make them apply for all applications I have put them in /etc/default/boinc-client which get loaded on startup:
$ grep GPU_ /etc/default/boinc-client GPU_MAX_ALLOC_PERCENT=95 GPU_MAX_HEAP_SIZE=95 export GPU_MAX_ALLOC_PERCENT export GPU_MAX_HEAP_SIZEHow-ever I end up with roughly the same message as before:
1 error detected in the compilation of "/tmp/OCLhykF21.cl".Frontend phase failed compilation.
OpenCL device has no FP64 support% Opening inputfile: ../../projects/einstein.phys.uwm.edu/LATeah1021L.dat
The error about not
)
The error about not supporting FP64 has nothing to do with the task crashing!
A few lines further down in the "stderr output", when the task has gotten to the real starting point, you will find:
This is the real cause of the task crashing.
As to what's causing the error I've no clue. *Sorry*
rickvanderzwet wrote:1 error
)
Holmis is quite correct. The above failure of a small test routine just confirms that your card has no FP64 support. All this should do is ensure that the follow-up stage at the end is performed using a CPU core.
As to why you get the subsequent error about allocating device memory, I have no idea either.
I did a quick google search and found this question and answers. I don't know if any of the comments there about 14.04.4 and 14.04.5 have any bearing on the matter. As I mentioned, I've never used Ubuntu so have no idea about how things are done in Ubuntu-land.
There was a final version of the catalyst package that (from memory) came out around mid 2015. In any case it was just before fglrx was deprecated due to the incompatible upgrade of the xorg package. I remember making sure that my machines using fglrx stayed with the previous version of xorg when they got that final version of fglrx. It actually gave a nice little performance improvement at the time.
Cheers,
Gary.
Holmis wrote:The error about
)
Thanks that's a really good suggestion! I was assuming the first error was the dominant one.
Found the cause of the memory allocation error. It's trying to allocate all available memory (268435456) where-as using the environment variables I told was only allowed to use 95% (Max allocation limit: 254803968). The code (quite) rightfully has no knowledge of the limit and thus does not honour them. I set the limit 100% and the task is running for a short while.
How-ever it is failing with an other error: