Simone Atzeni via llvm-dev
2017-Dec-26 22:37 UTC
[llvm-dev] Clang driver for OpenMP target offloading
Hi,
I am having some trouble to understand how the clang driver works for
OpenMP offloading.
For example, if I run the following command (with the -v flag):
clang -v -g -fopenmp offloading_success.c -o offloading_success
I can see the invocations that clang does and if I run those command one by
one I can successfully compile my program.
Now, if I do the same but with the offloading option, as in the following
command:
clang -v -g -fopenmp -fopenmp-targets=nvptx64-nvidia-cuda
offloading_success.c -o offloading_success
the command works, compiles correctly, and when I run the program, it
successfully run on the device.
However, if I show the invocations with "-v" and try to run the
commands
one by one it does not work and in particular I get an error with the last
command which is the linking part with "ld".
The error i get is the following:
/usr/bin/ld: cannot open linker script file /tmp/
offloading_success-2b4739.lk: No such file or directory
Where is this ".lk" file generated? Is there a missing commands in the
invocations?
I attached a file with the full invocation.
Thanks!
Best,
Simone
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20171226/03599906/attachment.html>
-------------- next part --------------
clang version 4.0.0 (https://github.com/clang-ykt/clang
4fdb5972bb1c16712aff9f752018f4f65ed7ac33) (https://github.com/clang-ykt/llvm
849ff0b41586f120c636e9cf9e106f4e16f7eed2)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/satzeni/usr_linux86-64/bin
Found candidate GCC installation: /usr/lib/gcc/i686-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/i686-redhat-linux/4.8.5
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.2
Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/4.8.5
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
Found CUDA installation: /usr/local/cuda, version 8.0
"/home/satzeni/usr_linux86-64_ykt/bin/clang-4.0" -cc1 -triple
x86_64-unknown-linux-gnu -emit-llvm-bc -emit-llvm-uselists -disable-free
-main-file-name offloading_success.c -mrelocation-model static -mthread-model
posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases
-munwind-tables -fuse-init-array -target-cpu x86-64 -v -dwarf-column-info
-debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -resource-dir
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0 -I
/home/satzeni/usr_linux86-64/include -internal-isystem /usr/local/include
-internal-isystem
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
-internal-externc-isystem /include -internal-externc-isystem /usr/include
-internal-isystem /usr/local/include -internal-isystem
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
-internal-externc-isystem /include -internal-externc-isystem /usr/include -O0
-fdebug-compilation-dir /home/satzeni/work/LLVM_ykt/test -ferror-limit 19
-fmessage-length 0 -fopenmp -fobjc-runtime=gcc -fdiagnostics-show-option -o
/tmp/offloading_success-374523.bc -x c offloading_success.c
-fopenmp-targets=nvptx64-nvidia-cuda
clang -cc1 version 4.0.0 based upon LLVM 4.0.0svn default target
x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory
"/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
/home/satzeni/usr_linux86-64/include
/usr/local/include
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
/usr/include
End of search list.
"/home/satzeni/usr_linux86-64_ykt/bin/clang-4.0" -cc1 -triple
nvptx64-nvidia-cuda -aux-triple x86_64-unknown-linux-gnu -S -disable-free
-main-file-name offloading_success.c -mrelocation-model pic -pic-level 2
-mthread-model posix -mdisable-fp-elim -fmath-errno -no-integrated-as
-fuse-init-array -mlink-cuda-bitcode
/usr/local/cuda/nvvm/libdevice/libdevice.compute_35.10.bc -target-feature +ptx42
-mlink-cuda-bitcode /home/satzeni/usr_linux86-64/lib/libomptarget-nvptx.bc
-target-cpu sm_35 -v -v -dwarf-column-info -debug-info-kind=limited
-dwarf-version=2 -debugger-tuning=cuda-gdb -resource-dir
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0 -I
/home/satzeni/usr_linux86-64/include -I /home/satzeni/usr_linux86-64/include
-internal-isystem /usr/local/include -internal-isystem
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
-internal-externc-isystem /include -internal-externc-isystem /usr/include
-internal-isystem /usr/local/include -internal-isystem
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
-internal-externc-isystem /include -internal-externc-isystem /usr/include -O0
-fno-dwarf-directory-asm -fdebug-compilation-dir
/home/satzeni/work/LLVM_ykt/test -ferror-limit 19 -fmessage-length 0 -fopenmp
-fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/offloading_success-1d7e72.s
-x c offloading_success.c -fopenmp-is-device -fopenmp-host-ir-file-path
/tmp/offloading_success-374523.bc
clang -cc1 version 4.0.0 based upon LLVM 4.0.0svn default target
x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
ignoring nonexistent directory "/include"
ignoring duplicate directory "/home/satzeni/usr_linux86-64/include"
ignoring duplicate directory "/usr/local/include"
ignoring duplicate directory
"/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include"
ignoring duplicate directory "/usr/include"
#include "..." search starts here:
#include <...> search starts here:
/home/satzeni/usr_linux86-64/include
/usr/local/include
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0/include
/usr/include
End of search list.
"/usr/local/cuda/bin/ptxas" -m64 -g --dont-merge-basicblocks
--return-at-end -v --gpu-name sm_35 --output-file
/tmp/offloading_success-6b6464.o /tmp/offloading_success-1d7e72.s -c
ptxas info : 12 bytes gmem
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE11DoneServingEjj
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext4PrivEi
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z24GetNumberOfWorkersInTeamv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z24GetNumberOfProcsInDevicev
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _ZL10atomicExchPyy
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE7DequeueEv
24 bytes stack frame, 24 bytes spill stores, 24 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE14DEQUEUE_TICKETEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr13ThreadsInTeamEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TeamDescr13InitTeamDescrEv
8 bytes stack frame, 8 bytes spill stores, 8 bytes spill loads
ptxas info : Function properties for __kmpc_kernel_deinit
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for __kmpc_kernel_init
32 bytes stack frame, 32 bytes spill stores, 32 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TeamDescr9WorkDescrEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr22InitLevelZeroTaskDescrEv
16 bytes stack frame, 12 bytes spill stores, 12 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE9IsServingEjj
8 bytes stack frame, 4 bytes spill stores, 4 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr17CopyFromWorkDescrEPS_
8 bytes stack frame, 8 bytes spill stores, 8 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext20SetTopLevelTaskDescrEiP25omptarget_nvptx_TaskDescr
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN28omptarget_nvptx_CounterGroup5ClearEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE14ENQUEUE_TICKETEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z18GetThreadIdInBlockv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z25GetNumberOfThreadsInBlockv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z19getMyWorkDescriptorv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext24InitThreadPrivateContextEi
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _ZL9atomicAddPyy_$_5
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Compiling entry function
'__omp_offloading_2b_155eb1f_main_l12' for 'sm_35'
ptxas info : Function properties for __omp_offloading_2b_155eb1f_main_l12
24 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Used 12 registers, 336 bytes cmem[0]
ptxas info : Function properties for _ZN25omptarget_nvptx_TaskDescr4CopyEPS_
16 bytes stack frame, 16 bytes spill stores, 16 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr11ThreadLimitEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _ZL10atomicExchPjj
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN28omptarget_nvptx_CounterGroup4InitERy
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr8NThreadsEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z13isGenericModev
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for __kmpc_kernel_parallel
32 bytes stack frame, 28 bytes spill stores, 28 bytes spill loads
ptxas info : Function properties for _Z25GetLogicalThreadIdInBlockv
8 bytes stack frame, 4 bytes spill stores, 4 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE2IDEj
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext15Level1TaskDescrEi
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TeamDescr18LevelZeroTaskDescrEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE7EnqueueEPS0_
24 bytes stack frame, 24 bytes spill stores, 24 bytes spill loads
ptxas info : Function properties for
_Z22setExecutionParameters13ExecutionMode11RuntimeMode
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
__omp_offloading_2b_155eb1f_main_l12_impl___debug__
32 bytes stack frame, 16 bytes spill stores, 16 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_WorkDescr13InitWorkDescrEv
8 bytes stack frame, 8 bytes spill stores, 8 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE10PopElementEj
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_WorkDescr12CounterGroupEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext20GetTopLevelTaskDescrEi
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z22GetNumberOfProcsInTeamv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN21omptarget_nvptx_QueueI36omptarget_nvptx_ThreadPrivateContextLj16EE11PushElementEjPS0_
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z4smidv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN36omptarget_nvptx_ThreadPrivateContext11TeamContextEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_WorkDescr13WorkTaskDescrEv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z19getMyTeamDescriptorv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _ZL9atomicAddPjj
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for __omp_kernel_initialization
40 bytes stack frame, 20 bytes spill stores, 20 bytes spill loads
ptxas info : Function properties for
_ZN25omptarget_nvptx_TaskDescr8CopyDataEPS_
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
ptxas info : Function properties for _Z17GetMasterThreadIDv
0 bytes stack frame, 0 bytes spill stores, 0 bytes spill loads
"/usr/bin/cp" /tmp/offloading_success-6b6464.o
/tmp/offloading_success-6b6464-6bf28c.cubin
"/usr/local/cuda/bin/nvlink" -o /tmp/offloading_success-f5cf9c.out -g
-v -arch sm_35 -L/home/satzeni/usr_linux86-64/lib -L/usr/local/cuda/lib64 -L.
-L/home/satzeni/usr_linux86-64_ykt/lib -lomptarget-nvptx
/tmp/offloading_success-6b6464-6bf28c.cubin
nvlink info : 87796543 bytes gmem
nvlink info : Function properties for
'__omp_offloading_2b_155eb1f_main_l12':
nvlink info : used 23 registers, 160 stack, 916 bytes smem, 336 bytes
cmem[0], 0 bytes lmem
"/home/satzeni/usr_linux86-64_ykt/bin/clang-4.0" -cc1 -triple
x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -main-file-name
offloading_success.c -mrelocation-model static -mthread-model posix
-mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases
-munwind-tables -fuse-init-array -target-cpu x86-64 -v -dwarf-column-info
-debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -resource-dir
/home/satzeni/usr_linux86-64_ykt/bin/../lib/clang/4.0.0 -O0
-fdebug-compilation-dir /home/satzeni/work/LLVM_ykt/test -ferror-limit 19
-fmessage-length 0 -fopenmp -fobjc-runtime=gcc -fdiagnostics-show-option -o
/tmp/offloading_success-bc3f15.o -x ir /tmp/offloading_success-374523.bc
clang -cc1 version 4.0.0 based upon LLVM 4.0.0svn default target
x86_64-unknown-linux-gnu
"/usr/bin/ld" --hash-style=gnu --no-add-needed --eh-frame-hdr -m
elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o offloading_success
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crt1.o
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crti.o
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o
-L/usr/lib/gcc/x86_64-redhat-linux/4.8.5
-L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64
-L/home/satzeni/usr_linux86-64_ykt/bin/../lib64 -L/lib/../lib64
-L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../..
-L/home/satzeni/usr_linux86-64_ykt/bin/../lib -L/lib -L/usr/lib
/tmp/offloading_success-bc3f15.o -lomp -lomptarget-nvptx
-L/home/satzeni/usr_linux86-64/lib -L/usr/local/cuda/lib64 -L. -lomp -lomptarget
-lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s
--no-as-needed /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/../../../../lib64/crtn.o -T
/tmp/offloading_success-333a1a.lk