David Demelier via llvm-dev
2019-Mar-25 12:52 UTC
[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution
Hello, I'm trying to create a pure LLVM toolchain (that will not depend on GNU and produce GNU-free code too) on a musl based distribution. For now, I use gcc to bootstrap and build all LLVM components. I do it individually because I was running out of space and memory trying to build all using LLVM_ENABLE_PROJECTS. Also, I don't want to create a all-in-one package. Then, once I'm able to build program with clang, I'll rebuild all using clang instead of gcc. # LLVM I've built LLVM using the following configuration: -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=On -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_HOST_TRIPLE=x86_64-linux-musl -DLLVM_ENABLE_RTTI=On # compiler-rt I've needed to disable xray/sanitizers otherwise it didn't build on musl. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DCOMPILER_RT_BUILD_BUILTINS=On -DCOMPILER_RT_BUILD_SANITIZERS=Off -DCOMPILER_RT_BUILD_XRAY=Off -DCOMPILER_RT_USE_BUILTINS_LIBRARY=On # libc++ -DBUILD_SHARED_LIBS=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DLIBCXX_HAS_MUSL_LIBC=On -DLIBCXX_HAS_GCC_S_LIB=Off -DLIBCXX_ENABLE_STATIC=Off # libc++abi -DBUILD_SHARED_LIBS=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DLIBCXXABI_ENABLE_STATIC=Off # libunwind -DBUILD_SHARED_LIBS=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DLIBUNWIND_ENABLE_STATIC=Off -DLIBUNWIND_USE_COMPILER_RT=On # clang And finally clang: -DBUILD_SHARED_LIBS=On -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr -DCLANG_DEFAULT_CXX_STDLIB=libc++ -DCLANG_DEFAULT_LINKER=lld -DCLANG_DEFAULT_RTLIB=compiler-rt -DCLANG_VENDOR="Vanilla Linux" Everything built fine, now the problem is the location of compiler-rt libraries which are not the correct place where clang/ld.lld searches. As the following output shows: clang -v /tmp/test.c Vanilla Linux clang version 8.0.0 (tags/RELEASE_800/final) (based on LLVM 8.0.0) Target: x86_64-unknown-linux-musl Thread model: posix InstalledDir: /usr/bin "/usr/bin/clang-8" -cc1 -triple x86_64-unknown-linux-musl -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir /usr/lib/clang/8.0.0 -internal-isystem /usr/local/include -internal-isystem /usr/lib/clang/8.0.0/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdebug-compilation-dir /usr/src/vanilla -ferror-limit 19 -fmessage-length 184 -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/test-1d99ce.o -x c /tmp/test.c -faddrsig clang -cc1 version 8.0.0 based upon LLVM 8.0.0 default target x86_64-linux-musl ignoring nonexistent directory "/include" #include "..." search starts here: #include <...> search starts here: /usr/local/include /usr/lib/clang/8.0.0/include /usr/include End of search list. "/usr/bin/ld.lld" -pie --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib/ld-musl-x86_64.so.1 -o a.out /usr/bin/../lib/Scrt1.o /usr/bin/../lib/crti.o crtbeginS.o -L/usr/bin/../lib -L/lib -L/usr/lib /tmp/test-1d99ce.o /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a crtendS.o /usr/bin/../lib/crtn.o ld.lld: error: cannot open crtbeginS.o: No such file or directory ld.lld: error: cannot open /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such file or directory ld.lld: error: cannot open /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such file or directory ld.lld: error: cannot open crtendS.o: No such file or directory clang-8: error: linker command failed with exit code 1 (use -v to see invocation) compiler-rt installed files under the following directories hierarchy instead: usr/lib/linux/libclang_rt.builtins-x86_64.a usr/lib/linux/libclang_rt.fuzzer-x86_64.a usr/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a usr/lib/linux/libclang_rt.profile-x86_64.a What did I misconfigured? Is it clang/lld or compiler-rt? Also do I need to do something else regarding crtbeginS.o and crtendS.o that are not provided by musl? Regards, -- David
Peter Smith via llvm-dev
2019-Mar-25 13:41 UTC
[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution
Hello David, I don't know much about the specifics of Musl, so I'm responding generally. As I understand it, clang expects to find the compiler-rt libraries relative to the resource directory, which you can find out the location of with clang --print-resource-dir . By default it is lib/clang/9.0.0 assuming you are building from master. I think that -DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that you would either need to take out the CMAKE_INSTALL_PREFIX or change the location of the resource directory, which I think that you can alter at build time. I don't think that there is a way of preventing clang adding crtbeginS.o and crtendS.o without --nostdlib or --nostartfiles but using these options will also not add the other crt*.o files that you may be expecting. There is a line in lib/Driver/Toolchains/Gnu.cpp which the Linux driver uses: const bool HasCRTBeginEndFiles ToolChain.getTriple().hasEnvironment() || (ToolChain.getTriple().getVendor() != llvm::Triple::MipsTechnologies); It looks like Musl may need to be included there? Peter On Mon, 25 Mar 2019 at 12:52, David Demelier via llvm-dev <llvm-dev at lists.llvm.org> wrote:> > Hello, > > I'm trying to create a pure LLVM toolchain (that will not depend on GNU > and produce GNU-free code too) on a musl based distribution. > > For now, I use gcc to bootstrap and build all LLVM components. I do it > individually because I was running out of space and memory trying to > build all using LLVM_ENABLE_PROJECTS. Also, I don't want to create a > all-in-one package. Then, once I'm able to build program with clang, > I'll rebuild all using clang instead of gcc. > > # LLVM > > I've built LLVM using the following configuration: > > -DCMAKE_BUILD_TYPE=Release > -DBUILD_SHARED_LIBS=On > -DCMAKE_INSTALL_PREFIX=/usr > -DLLVM_HOST_TRIPLE=x86_64-linux-musl > -DLLVM_ENABLE_RTTI=On > > # compiler-rt > > I've needed to disable xray/sanitizers otherwise it didn't build on musl. > > -DCMAKE_BUILD_TYPE=Release > -DCMAKE_INSTALL_PREFIX=/usr > -DCOMPILER_RT_BUILD_BUILTINS=On > -DCOMPILER_RT_BUILD_SANITIZERS=Off > -DCOMPILER_RT_BUILD_XRAY=Off > -DCOMPILER_RT_USE_BUILTINS_LIBRARY=On > > # libc++ > > -DBUILD_SHARED_LIBS=On > -DCMAKE_BUILD_TYPE=Release > -DCMAKE_INSTALL_PREFIX=/usr > -DLIBCXX_HAS_MUSL_LIBC=On > -DLIBCXX_HAS_GCC_S_LIB=Off > -DLIBCXX_ENABLE_STATIC=Off > > # libc++abi > > -DBUILD_SHARED_LIBS=On > -DCMAKE_BUILD_TYPE=Release > -DCMAKE_INSTALL_PREFIX=/usr > -DLIBCXXABI_ENABLE_STATIC=Off > > # libunwind > > -DBUILD_SHARED_LIBS=On > -DCMAKE_BUILD_TYPE=Release > -DCMAKE_INSTALL_PREFIX=/usr > -DLIBUNWIND_ENABLE_STATIC=Off > -DLIBUNWIND_USE_COMPILER_RT=On > > # clang > > And finally clang: > > -DBUILD_SHARED_LIBS=On > -DCMAKE_BUILD_TYPE=Release > -DCMAKE_INSTALL_PREFIX=/usr > -DCLANG_DEFAULT_CXX_STDLIB=libc++ > -DCLANG_DEFAULT_LINKER=lld > -DCLANG_DEFAULT_RTLIB=compiler-rt > -DCLANG_VENDOR="Vanilla Linux" > > Everything built fine, now the problem is the location of compiler-rt > libraries which are not the correct place where clang/ld.lld searches. > As the following output shows: > > clang -v /tmp/test.c > Vanilla Linux clang version 8.0.0 (tags/RELEASE_800/final) (based on > LLVM 8.0.0) > Target: x86_64-unknown-linux-musl > Thread model: posix > InstalledDir: /usr/bin > "/usr/bin/clang-8" -cc1 -triple x86_64-unknown-linux-musl -emit-obj > -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names > -main-file-name test.c -mrelocation-model pic -pic-level 2 -pic-is-pie > -mthread-model posix -mdisable-fp-elim -fmath-errno -masm-verbose > -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu > x86-64 -dwarf-column-info -debugger-tuning=gdb -v -resource-dir > /usr/lib/clang/8.0.0 -internal-isystem /usr/local/include > -internal-isystem /usr/lib/clang/8.0.0/include -internal-externc-isystem > /include -internal-externc-isystem /usr/include -fdebug-compilation-dir > /usr/src/vanilla -ferror-limit 19 -fmessage-length 184 > -fobjc-runtime=gcc -fdiagnostics-show-option -o /tmp/test-1d99ce.o -x c > /tmp/test.c -faddrsig > clang -cc1 version 8.0.0 based upon LLVM 8.0.0 default target > x86_64-linux-musl > ignoring nonexistent directory "/include" > #include "..." search starts here: > #include <...> search starts here: > /usr/local/include > /usr/lib/clang/8.0.0/include > /usr/include > End of search list. > "/usr/bin/ld.lld" -pie --eh-frame-hdr -m elf_x86_64 -dynamic-linker > /lib/ld-musl-x86_64.so.1 -o a.out /usr/bin/../lib/Scrt1.o > /usr/bin/../lib/crti.o crtbeginS.o -L/usr/bin/../lib -L/lib -L/usr/lib > /tmp/test-1d99ce.o > /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lc > /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a crtendS.o > /usr/bin/../lib/crtn.o > ld.lld: error: cannot open crtbeginS.o: No such file or directory > ld.lld: error: cannot open > /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such > file or directory > ld.lld: error: cannot open > /usr/lib/clang/8.0.0/lib/linux/libclang_rt.builtins-x86_64.a: No such > file or directory > ld.lld: error: cannot open crtendS.o: No such file or directory > clang-8: error: linker command failed with exit code 1 (use -v to see > invocation) > > compiler-rt installed files under the following directories hierarchy > instead: > > usr/lib/linux/libclang_rt.builtins-x86_64.a > usr/lib/linux/libclang_rt.fuzzer-x86_64.a > usr/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a > usr/lib/linux/libclang_rt.profile-x86_64.a > > What did I misconfigured? Is it clang/lld or compiler-rt? Also do I need > to do something else regarding crtbeginS.o and crtendS.o that are not > provided by musl? > > Regards, > > -- > David > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
David Demelier via llvm-dev
2019-Mar-25 14:05 UTC
[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution
Le 25/03/2019 à 14:41, Peter Smith a écrit :> Hello David, > > I don't know much about the specifics of Musl, so I'm responding generally. > > As I understand it, clang expects to find the compiler-rt libraries > relative to the resource directory, which you can find out the > location of with clang --print-resource-dir . By default it is > lib/clang/9.0.0 assuming you are building from master. I think that > -DCMAKE_INSTALL_PREFIX=/usr has broken that assumption. I think that > you would either need to take out the CMAKE_INSTALL_PREFIX or change > the location of the resource directory, which I think that you can > alter at build time. >You're right, I've just checked both packages in Alpine and Arch and they indeed move the /usr/lib/linux directory under the clang's one. I've fixed that it worked, now I still have to find the solution regarding crtbeginS.so and crtendS.so. Thanks :)> I don't think that there is a way of preventing clang adding > crtbeginS.o and crtendS.o without --nostdlib or --nostartfiles but > using these options will also not add the other crt*.o files that you > may be expecting. > > There is a line in lib/Driver/Toolchains/Gnu.cpp which the Linux driver uses: > const bool HasCRTBeginEndFiles > ToolChain.getTriple().hasEnvironment() || > (ToolChain.getTriple().getVendor() != llvm::Triple::MipsTechnologies); > It looks like Musl may need to be included there?I'll have a look, I don't know much the lowest part of the toolchain build process so I don't understand very much what's going on there. Hopefully someone that has better low-level knowledges and use musl will be able to guide me for this. What can I tell is that musl provide those files: usr/lib/Scrt1.o usr/lib/crt1.o usr/lib/crti.o usr/lib/crtn.o usr/lib/ld-musl-x86_64.so.1 Regards, -- David
Andrew Kelley via llvm-dev
2019-Mar-25 20:24 UTC
[llvm-dev] Trying to create a pure LLVM toolchain on musl based distribution
On 3/25/19 8:52 AM, David Demelier via llvm-dev wrote:> Hello, > > I'm trying to create a pure LLVM toolchain (that will not depend on GNU > and produce GNU-free code too) on a musl based distribution. > > # LLVM > # compiler-rt > # libc++ > # libc++abi > # libunwind > # clangDavid, You may be interested to know that Zig (https://ziglang.org/) ships with such a toolchain already. It cross compiles components (musl, libunwind, etc) for the target lazily from source, and supports these libc targets: aarch64_be-linux-gnu aarch64_be-linux-musl aarch64-linux-gnu aarch64-linux-musleabi armeb-linux-gnueabi armeb-linux-gnueabihf armeb-linux-musleabi armeb-linux-musleabihf arm-linux-gnueabi arm-linux-gnueabihf arm-linux-musleabi arm-linux-musleabihf i386-linux-gnu i386-linux-musl mips64el-linux-gnuabi64 mips64el-linux-gnuabin32 mips64el-linux-musl mips64-linux-gnuabi64 mips64-linux-gnuabin32 mips64-linux-musl mipsel-linux-gnu mipsel-linux-musl mips-linux-gnu mips-linux-musl powerpc64le-linux-gnu powerpc64le-linux-musl powerpc64-linux-gnu powerpc64-linux-musl powerpc-linux-gnu powerpc-linux-musl riscv32-linux-musl riscv64-linux-gnu riscv64-linux-musl s390x-linux-gnu s390x-linux-musl sparc-linux-gnu sparcv9-linux-gnu x86_64-linux-gnu x86_64-linux-gnux32 x86_64-linux-musl Here's an example of using Zig to compile hello-world.c for aarch64-linux-gnu: andy at xps ~/m/z/hello> cat hello.c #include <stdio.h> int main(int argc, char **argv) { printf("what up dawg\n"); return 0; } andy at xps ~/m/z/hello> zig build-exe --c-source hello.c --library c -target aarch64v8-linux-gnu andy at xps ~/m/z/hello> file hello hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 2.0.0, with debug_info, not stripped And now for x86_64-linux-musl: andy at xps ~/m/z/hello> zig build-exe --c-source hello.c --library c -target x86_64-linux-musl andy at xps ~/m/z/hello> ./hello what up dawg andy at xps ~/m/z/hello> ldd ./hello not a dynamic executable This works on any OS including Windows and macOS since it has no system dependencies. This feature is new and not well tested yet, but if it piques your interest, you can play with this functionality using pre-built binaries available on https://ziglang.org/download/ or you can build Zig master branch from source. Regards, Andrew -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190325/64be6724/attachment.sig>