> Alexey's approach with CMake sub-projects.I prefer that direction as well, but what I've proposed is a solution that works today. To support cross-compilation, we'll need to loop over each supported arch (llvm-config --targets-built), then loop over each supported triple for each arch (hard-coded map?), and then pair up each triple with a sysroot (system paths provided by the user).> I think it already copies > compiler-rt build products to the parent build directory, right?The Android compiler-rt build does this, but in an unnecessarily complex way. My proposal is a simplification of that process. What is described in "llvm/cmake/platforms/Android.cmake" is to build llvm+clang for the host architecture and then run a second build to cross-compile llvm+compiler-rt to get a the ASan shared object and test suite. Instead, I propose building only llvm for the host architecture (for the purpose of building llvm-config) and then only compile compiler-rt for Android. No need to build clang or to cross-compile llvm.> Requiring that llvm is installed complicates things.Sorry for the confusion. By "llvm install directory", I don't mean to install llvm to the root directory. I'm referring to the intermediary directory pointed to by CMAKE_INSTALL_PREFIX. When you run "ninja install", the build populates this directory with only the libs, headers, docs, and executables that the LLVM build intends customers to use directly. In the case of compiler-rt, we need LLVM's "bin/llvm-config" to determine the build mode, flags, and targets available.> This should not be too hard to fix.Porting the LLVM build to Android? That seems like a more challenging solution than to cut out the unused cross-compiled llvm build. -Greg On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov <eugeni.stepanov at gmail.com> wrote:> I'd prefer something based on Alexey's approach with CMake > sub-projects. I think it already copies compiler-rt build products to > the parent build directory, right? > > Requiring that llvm is installed complicates things. > > On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald <garious at gmail.com> wrote: >> The build for the Android sanitizers is unique in that it needs to >> link against the Android system libraries to create a shared object >> and its test suites. The current solution to build ASan is to drop >> the compiler-rt repo into the llvm source tree and cross-compile the >> llvm build for Android. This is a bit awkward for few reasons: >> >> 1) Not all of llvm can be cross-compiled for Android. "ninja all" >> causes expected build errors. > > This should not be too hard to fix. > >> 2) The sanitizers don't depend on any LLVM libraries, only llvm-config >> for its build configuration. >> 3) No "install" rule. Instead, you cherry-pick files from the build directory. >> >> Building against the LLVM install directory cleans this up nicely and >> as it turns out, this mostly works today (see CMake configuration >> below). The only missing pieces are that the shared object is not >> added to the install directory and the test suites are not built. Is >> this a build configuration you'd consider using? >> >> $ cmake -G Ninja .. \ >> -DCMAKE_INSTALL_PREFIX=ship \ >> -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \ >> -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \ >> -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \ >> -DANDROID=1 \ >> -DCMAKE_SYSTEM_NAME=Linux \ >> -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm \ >> -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm >> ... >> $ ninja install >> ... >> [68/68] >> -- Install configuration: "Release" >> -- Installing: ship/include/sanitizer/asan_interface.h >> -- Installing: ship/include/sanitizer/common_interface_defs.h >> -- Installing: ship/include/sanitizer/dfsan_interface.h >> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h >> -- Installing: ship/include/sanitizer/lsan_interface.h >> -- Installing: ship/include/sanitizer/msan_interface.h >> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h >> -- Installing: ship/asan_blacklist.txt >> -- Installing: ship/bin/asan_device_setup >> >> Thanks, >> Greg
On Thu, Mar 27, 2014 at 10:18 PM, Greg Fitzgerald <garious at gmail.com> wrote:>> Alexey's approach with CMake sub-projects. > > I prefer that direction as well, but what I've proposed is a solution > that works today. To support cross-compilation, we'll need to loop > over each supported arch (llvm-config --targets-built), then loop over > each supported triple for each arch (hard-coded map?), and then pair > up each triple with a sysroot (system paths provided by the user). > > >> I think it already copies >> compiler-rt build products to the parent build directory, right? > > The Android compiler-rt build does this, but in an unnecessarily > complex way. My proposal is a simplification of that process. What > is described in "llvm/cmake/platforms/Android.cmake" is to build > llvm+clang for the host architecture and then run a second build to > cross-compile llvm+compiler-rt to get a the ASan shared object and > test suite. Instead, I propose building only llvm for the host > architecture (for the purpose of building llvm-config) and then only > compile compiler-rt for Android. No need to build clang or to > cross-compile llvm. > > >> Requiring that llvm is installed complicates things. > > Sorry for the confusion. By "llvm install directory", I don't mean to > install llvm to the root directory. I'm referring to the intermediary > directory pointed to by CMAKE_INSTALL_PREFIX. When you run "ninja > install", the build populates this directory with only the libs, > headers, docs, and executables that the LLVM build intends customers > to use directly. In the case of compiler-rt, we need LLVM's > "bin/llvm-config" to determine the build mode, flags, and targets > available.Yes, so "ninja install" is optional. This should work with LLVM build directory just as well.> > >> This should not be too hard to fix. > > Porting the LLVM build to Android? That seems like a more challenging > solution than to cut out the unused cross-compiled llvm build.Note that ASan tests on Android require llvm-symbolizer binary. Building just compiler-rt for the target would not be enough.> > -Greg > > On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov > <eugeni.stepanov at gmail.com> wrote: >> I'd prefer something based on Alexey's approach with CMake >> sub-projects. I think it already copies compiler-rt build products to >> the parent build directory, right? >> >> Requiring that llvm is installed complicates things. >> >> On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald <garious at gmail.com> wrote: >>> The build for the Android sanitizers is unique in that it needs to >>> link against the Android system libraries to create a shared object >>> and its test suites. The current solution to build ASan is to drop >>> the compiler-rt repo into the llvm source tree and cross-compile the >>> llvm build for Android. This is a bit awkward for few reasons: >>> >>> 1) Not all of llvm can be cross-compiled for Android. "ninja all" >>> causes expected build errors. >> >> This should not be too hard to fix. >> >>> 2) The sanitizers don't depend on any LLVM libraries, only llvm-config >>> for its build configuration. >>> 3) No "install" rule. Instead, you cherry-pick files from the build directory. >>> >>> Building against the LLVM install directory cleans this up nicely and >>> as it turns out, this mostly works today (see CMake configuration >>> below). The only missing pieces are that the shared object is not >>> added to the install directory and the test suites are not built. Is >>> this a build configuration you'd consider using? >>> >>> $ cmake -G Ninja .. \ >>> -DCMAKE_INSTALL_PREFIX=ship \ >>> -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \ >>> -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \ >>> -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \ >>> -DANDROID=1 \ >>> -DCMAKE_SYSTEM_NAME=Linux \ >>> -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm \ >>> -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm >>> ... >>> $ ninja install >>> ... >>> [68/68] >>> -- Install configuration: "Release" >>> -- Installing: ship/include/sanitizer/asan_interface.h >>> -- Installing: ship/include/sanitizer/common_interface_defs.h >>> -- Installing: ship/include/sanitizer/dfsan_interface.h >>> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h >>> -- Installing: ship/include/sanitizer/lsan_interface.h >>> -- Installing: ship/include/sanitizer/msan_interface.h >>> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h >>> -- Installing: ship/asan_blacklist.txt >>> -- Installing: ship/bin/asan_device_setup >>> >>> Thanks, >>> Greg
> Note that ASan tests on Android require llvm-symbolizer binary.That's a really good point. And I see that llvm-symbolizer can't just be pulled into compiler-rt because it has dependencies on DebugInfo, Object, and Support libraries. This throws a big wrench in Alexey's plan to have the native compiler-rt build generate the cross-compiled binaries for all supported targets. We would need to do the same in the llvm repo. The alternative is to run a separate cross-compiled build for each architecture. :-/> Yes, so "ninja install" is optional.But the 'install' depends on the 'all' target. -Greg On Fri, Mar 28, 2014 at 12:39 AM, Evgeniy Stepanov <eugeni.stepanov at gmail.com> wrote:> On Thu, Mar 27, 2014 at 10:18 PM, Greg Fitzgerald <garious at gmail.com> wrote: >>> Alexey's approach with CMake sub-projects. >> >> I prefer that direction as well, but what I've proposed is a solution >> that works today. To support cross-compilation, we'll need to loop >> over each supported arch (llvm-config --targets-built), then loop over >> each supported triple for each arch (hard-coded map?), and then pair >> up each triple with a sysroot (system paths provided by the user). >> >> >>> I think it already copies >>> compiler-rt build products to the parent build directory, right? >> >> The Android compiler-rt build does this, but in an unnecessarily >> complex way. My proposal is a simplification of that process. What >> is described in "llvm/cmake/platforms/Android.cmake" is to build >> llvm+clang for the host architecture and then run a second build to >> cross-compile llvm+compiler-rt to get a the ASan shared object and >> test suite. Instead, I propose building only llvm for the host >> architecture (for the purpose of building llvm-config) and then only >> compile compiler-rt for Android. No need to build clang or to >> cross-compile llvm. >> >> >>> Requiring that llvm is installed complicates things. >> >> Sorry for the confusion. By "llvm install directory", I don't mean to >> install llvm to the root directory. I'm referring to the intermediary >> directory pointed to by CMAKE_INSTALL_PREFIX. When you run "ninja >> install", the build populates this directory with only the libs, >> headers, docs, and executables that the LLVM build intends customers >> to use directly. In the case of compiler-rt, we need LLVM's >> "bin/llvm-config" to determine the build mode, flags, and targets >> available. > > Yes, so "ninja install" is optional. This should work with LLVM build > directory just as well. > >> >> >>> This should not be too hard to fix. >> >> Porting the LLVM build to Android? That seems like a more challenging >> solution than to cut out the unused cross-compiled llvm build. > > Note that ASan tests on Android require llvm-symbolizer binary. > Building just compiler-rt for the target would not be enough. > >> >> -Greg >> >> On Thu, Mar 27, 2014 at 1:14 AM, Evgeniy Stepanov >> <eugeni.stepanov at gmail.com> wrote: >>> I'd prefer something based on Alexey's approach with CMake >>> sub-projects. I think it already copies compiler-rt build products to >>> the parent build directory, right? >>> >>> Requiring that llvm is installed complicates things. >>> >>> On Thu, Mar 27, 2014 at 5:00 AM, Greg Fitzgerald <garious at gmail.com> wrote: >>>> The build for the Android sanitizers is unique in that it needs to >>>> link against the Android system libraries to create a shared object >>>> and its test suites. The current solution to build ASan is to drop >>>> the compiler-rt repo into the llvm source tree and cross-compile the >>>> llvm build for Android. This is a bit awkward for few reasons: >>>> >>>> 1) Not all of llvm can be cross-compiled for Android. "ninja all" >>>> causes expected build errors. >>> >>> This should not be too hard to fix. >>> >>>> 2) The sanitizers don't depend on any LLVM libraries, only llvm-config >>>> for its build configuration. >>>> 3) No "install" rule. Instead, you cherry-pick files from the build directory. >>>> >>>> Building against the LLVM install directory cleans this up nicely and >>>> as it turns out, this mostly works today (see CMake configuration >>>> below). The only missing pieces are that the shared object is not >>>> added to the install directory and the test suites are not built. Is >>>> this a build configuration you'd consider using? >>>> >>>> $ cmake -G Ninja .. \ >>>> -DCMAKE_INSTALL_PREFIX=ship \ >>>> -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc \ >>>> -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ \ >>>> -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship \ >>>> -DANDROID=1 \ >>>> -DCMAKE_SYSTEM_NAME=Linux \ >>>> -DCMAKE_C_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm \ >>>> -DCMAKE_CXX_FLAGS=--sysroot=$(ndkDir)/platforms/android-19/arch-arm >>>> ... >>>> $ ninja install >>>> ... >>>> [68/68] >>>> -- Install configuration: "Release" >>>> -- Installing: ship/include/sanitizer/asan_interface.h >>>> -- Installing: ship/include/sanitizer/common_interface_defs.h >>>> -- Installing: ship/include/sanitizer/dfsan_interface.h >>>> -- Installing: ship/include/sanitizer/linux_syscall_hooks.h >>>> -- Installing: ship/include/sanitizer/lsan_interface.h >>>> -- Installing: ship/include/sanitizer/msan_interface.h >>>> -- Installing: ship/include/sanitizer/tsan_interface_atomic.h >>>> -- Installing: ship/asan_blacklist.txt >>>> -- Installing: ship/bin/asan_device_setup >>>> >>>> Thanks, >>>> Greg