Compiling bitcode header

[ghostanon]

I am hoping to find some direction or preferably actual help. I've finally ran out of talent when it comes to Linux toubleshooting. As some may know I have been struggling to get OpenCL to work in my AMD 7900xtx. I think I have finally figured out the issue:

=== CL_PROGRAM_BUILD_LOG ===
fatal error: cannot open file '/usr/lib/clc/gfx1100-amdgcn-mesa-mesa3d.bc': file doesn't start with bitcode header

And after pouring through all the files I have downloaded trying to get something to work I noticed these:
see attached
I don't know for sure but I suspect that they have something to do with either creating or compiling the file I need. There's alot more files of all sorts and it seems like it should all come together but it won't. I've updated, rebooted, installed all the necessary repo's along with gpg signatures. I've even tied copying the files into the other one out of sheer desperation. If anyone is willing to help, private message me. I'm working from home for a couple days so it won't take me long toreply. Thanks in advance.

 

[f33dm3bits]

I think I found something that may be of use to use.

Release AOMP Release 16.0-1 · ROCm/aomp

These are the release notes for AOMP 16.0-1. This release uses modifications to the LLVM development trunk called the "amd-stg-open" branch. This is found at https://github.com/RadeonOpenCompute/ll...

github.com

The changes from 16.0-0 to 16.0-1 include:

• Enhanced xteam reductions, no codegen.
• Optimized wait for signals (perf gain).
• Fix aompcc and mark for deprecation.
• Added switch for code object version 5. Version 4 is still default.
• Support for gfx1100 - gfx1103.
• Support for order(concurrent).
• Build OpenMP warnings cleaned up.
• Support atomic min/max on MI200.
• Support for device new and delete.
• Bumped cmake version to 3.18.5.
• Switch to ROCm 5.3 sources.

I actually installed it on my Kali vm using the instructions they give.

aomp/docs/UBUNTUINSTALL.md at aomp-dev · ROCm/aomp

AOMP is an open source Clang/LLVM based compiler with added support for the OpenMP® API on Radeon™ GPUs. Use this repository for releases, issues, documentation, packaging, and examples. - ROCm...

github.com

Then I get the following files.

locate gfx1100
/usr/lib/aomp_17.0-2/lib/libomptarget-old-amdgpu-gfx1100.bc
/usr/lib/aomp_17.0-2/lib/libdevice/libhostexec-amdgcn-gfx1100.bc
/usr/lib/aomp_17.0-2/lib-debug/libomptarget-old-amdgpu-gfx1100.bc
/usr/lib/aomp_17.0-2/lib-debug/libdevice/libhostexec-amdgcn-gfx1100.bc
/usr/lib/aomp_17.0-2/test/roctracer/gfx1100_copy.hsaco

May you can work with one of those files?
Also I cloned the llvm-project to that same vm, look at the files that are in the libclc directory.

[~/Downloads/git/llvm-project/libclc]
└─$ ls -tlr           
total 108
-rw-r--r-- 1 kali kali  1612 Jul  8 09:18 README.TXT
-rw-r--r-- 1 kali kali 16469 Jul  8 09:18 LICENSE.TXT
-rw-r--r-- 1 kali kali    42 Jul  8 09:18 CREDITS.TXT
-rw-r--r-- 1 kali kali 11957 Jul  8 09:18 CMakeLists.txt
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 clspv
-rwxr-xr-x 1 kali kali   531 Jul  8 09:18 check_external_calls.sh
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgpu
lrwxrwxrwx 1 kali kali    13 Jul  8 09:18 amdgcn-mesa3d -> amdgcn-amdhsa
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgcn-amdhsa
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgcn
-rwxr-xr-x 1 kali kali   224 Jul  8 09:18 compile-test.sh
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 cmake
lrwxrwxrwx 1 kali kali     5 Jul  8 09:18 clspv64 -> clspv
drwxr-xr-x 4 kali kali  4096 Jul  8 09:18 generic
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 ptx-nvidiacl
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 ptx
-rw-r--r-- 1 kali kali   281 Jul  8 09:18 libclc.pc.in
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 www
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 utils
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 test
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 spirv64
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 spirv
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 r600

Yeah that one file:
lrwxrwxrwx 1 kali kali 13 Jul 8 09:18 amdgcn-mesa3d -> amdgcn-amdhsa

Maybe if you figure out how to compile the libclc version from llvm-project git repo. you will end up with the correct binary that you are currently missing(gfx1100-amdgcn-mesa-mesa3d.bc)?

 

[ghostanon]

Anyone have an idea how long this will take. I went and compiled the OpenCL using git hub and CMake. Now I'm just sitting here.


host@oracle:~/CLmain.c/vcpkg$ cmake --build ./build --config Release
[ 50%] Building C object CMakeFiles/HelloOpenCL.dir/Main.c.o
[100%] Linking C executable HelloOpenCL
[100%] Built target HelloOpenCL

 

[ghostanon]

I tried the ROCm and it just wouldn't play ball. There was a warning saying some distros don't accept it very well. Btw, it nothing short of a miracle and some copy and paste that I got this done. This supposed to update from 1.2 >3.0, I imagine that will include support for gfx11.

 

[ghostanon]

Ok, I went through and did exactly as you posted, I still get this.

=== CL_PROGRAM_BUILD_LOG ===
fatal error: cannot open file '/usr/lib/clc/gfx1100-amdgcn-mesa-mesa3d.bc': file doesn't start with bitcode header

Which is really hard to explain and I don't even know where to begin because when I enter locate gfx1100 I get the following: Like what the actual f*%k is going on? I'm really starting to get pissed now.


ghost@oracle:~$ locate gfx1100
/home/ghost/Downloads/clang+llvm-16.0.0-amd64-unknown-freebsd13/lib/libomptarget-amdgpu-gfx1100.bc
/home/ghost/Downloads/clangllvm.sh/lib/libomptarget-amdgpu-gfx1100.bc
/home/ghost/Downloads/mesa-23.1.3/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/home/ghost/Downloads/mesa-23.1.3/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/home/ghost/Downloads/mesa-23.1.3/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/home/ghost/Downloads/mesa-gfx11-little-stuff-reviewed1/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/home/ghost/Downloads/mesa-gfx11-little-stuff-reviewed1/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/home/ghost/Downloads/mesa-gfx11-little-stuff-reviewed1/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/home/ghost/Downloads/mesa-main/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/home/ghost/Downloads/mesa-main/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/home/ghost/Downloads/mesa-main/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/home/ghost/mesa/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/home/ghost/mesa/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/home/ghost/mesa/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/home/ghost/mesa2/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/home/ghost/mesa2/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/home/ghost/mesa2/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/usr/lib/clc/gfx1100-amdgcn-mesa-mesa3d.bc
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1 (2)/src/amd/ci/external/radv-gfx1100-aco-fails.txt
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1 (2)/src/amd/ci/external/radv-gfx1100-aco-flakes.txt
/usr/lib/clc/mesa-gfx11-little-stuff-reviewed1 (2)/src/gallium/drivers/radeonsi/ci/gfx11-gfx1100-fail.csv

 

[ghostanon]

I just tried one last thing. I deleted the file that didn't have the bitcode header. So now there's no gfx1100 in there so maybe it will recognize that and populate it with the correct one.

 

[ghostanon]

I think I found something that may be of use to use.

Release AOMP Release 16.0-1 · ROCm/aomp

These are the release notes for AOMP 16.0-1. This release uses modifications to the LLVM development trunk called the "amd-stg-open" branch. This is found at https://github.com/RadeonOpenCompute/ll...

github.com

I actually installed it on my Kali vm using the instructions they give.

aomp/docs/UBUNTUINSTALL.md at aomp-dev · ROCm/aomp

AOMP is an open source Clang/LLVM based compiler with added support for the OpenMP® API on Radeon™ GPUs. Use this repository for releases, issues, documentation, packaging, and examples. - ROCm...

github.com

Then I get the following files.

locate gfx1100
/usr/lib/aomp_17.0-2/lib/libomptarget-old-amdgpu-gfx1100.bc
/usr/lib/aomp_17.0-2/lib/libdevice/libhostexec-amdgcn-gfx1100.bc
/usr/lib/aomp_17.0-2/lib-debug/libomptarget-old-amdgpu-gfx1100.bc
/usr/lib/aomp_17.0-2/lib-debug/libdevice/libhostexec-amdgcn-gfx1100.bc
/usr/lib/aomp_17.0-2/test/roctracer/gfx1100_copy.hsaco

May you can work with one of those files?
Also I cloned the llvm-project to that same vm, look at the files that are in the libclc directory.

[~/Downloads/git/llvm-project/libclc]
└─$ ls -tlr          
total 108
-rw-r--r-- 1 kali kali  1612 Jul  8 09:18 README.TXT
-rw-r--r-- 1 kali kali 16469 Jul  8 09:18 LICENSE.TXT
-rw-r--r-- 1 kali kali    42 Jul  8 09:18 CREDITS.TXT
-rw-r--r-- 1 kali kali 11957 Jul  8 09:18 CMakeLists.txt
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 clspv
-rwxr-xr-x 1 kali kali   531 Jul  8 09:18 check_external_calls.sh
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgpu
lrwxrwxrwx 1 kali kali    13 Jul  8 09:18 amdgcn-mesa3d -> amdgcn-amdhsa
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgcn-amdhsa
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 amdgcn
-rwxr-xr-x 1 kali kali   224 Jul  8 09:18 compile-test.sh
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 cmake
lrwxrwxrwx 1 kali kali     5 Jul  8 09:18 clspv64 -> clspv
drwxr-xr-x 4 kali kali  4096 Jul  8 09:18 generic
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 ptx-nvidiacl
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 ptx
-rw-r--r-- 1 kali kali   281 Jul  8 09:18 libclc.pc.in
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 www
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 utils
drwxr-xr-x 2 kali kali  4096 Jul  8 09:18 test
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 spirv64
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 spirv
drwxr-xr-x 3 kali kali  4096 Jul  8 09:18 r600

Yeah that one file:
lrwxrwxrwx 1 kali kali 13 Jul 8 09:18 amdgcn-mesa3d -> amdgcn-amdhsa

Maybe if you figure out how to compile the libclc version from llvm-project git repo. you will end up with the correct binary that you are currently missing(gfx1100-amdgcn-mesa-mesa3d.bc)?

I finally got it. It took me all day and I think I downloaded the entire internet. It ending up being that I needed the amdGCN not gpu drivers. The only down side is that I downloaded so much stuff that after one little session on debian.org it downloaded well, everything I think. Then for some strange reason when I did the update and upgade, it was upgrading programs in the hundreds and also removing ones that I couldn't figure out why. It even took the settings for my terminal, now it looks like crap. I leave a little example of of hashcat cracking 499 SHA1 hashed passwords.

 

[f33dm3bits]

I finally got it. It took me all day and I think I downloaded the entire internet. It ending up being that I needed the amdGCN not gpu drivers.

So now you have a working "/usr/lib/clc/gfx1100-amdgcn-mesa-mesa3d.bc'" file, hashcat sees the expected devices and works as you would expect it to?

 

[ghostanon]

Yeah no only that but when I do clinfo there's 2 platforms that actually have gfx1100 listed as the vendor name. I found a the amdgcn I ended up needing in the debian repository. After I pulled them off the site and started downloading that's when all hell broke loose. Now I don't know what it needs and doesn't so I'm just archiving everything.

 

[ghostanon]

Btw, Actually here's what is left in that afile. I don't know where everything else went and don't know where the file I was missing is. And I don't care, I'm over it.