llvm dev - Mar 2017 - Use of host/target compiler when building compiler-rt

On Thu, Mar 9, 2017 at 11:25 AM Chris Bieneman <beanz at apple.com> wrote:
> On Mar 8, 2017, at 4:42 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
>
>
> On Wed, Mar 8, 2017 at 3:23 PM Chris Bieneman <beanz at apple.com>
wrote:
>
> On Mar 8, 2017, at 3:16 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
>
>
> On Wed, Mar 8, 2017 at 2:55 PM Chris Bieneman <beanz at apple.com>
wrote:
>
> David,
>
> This is an area that has had a lot of development over the last two years.
>
> There are two supported ways in the LLVM build system to build compiler-rt
> with the just-built compiler.
>
> 1) The legacy way is for if compiler-rt is under LLVM/projects. You can
> specify -DLLVM_BUILD_EXTERNAL_COMPILER_RT=On, which will configure
> compiler-rt using the just-built clang after clang is built.
>
>
> I thought the BUILD_EXTERNAL variables were for use when projects were not
> embedded within the llvm source tree (mostly in use by Takumi's flat
> buildbots that checout the top-level project without embedding, say, clang
> or compiler-rt within the llvm source tree)?
>
>
> You are confusing this with the similarly
> named LLVM_EXTERNAL_${nameUPPER}_SOURCE_DIR variables.
>
>
> Ah, right - indeed.
>
>
> 2) The new way, is to place compiler-rt under LLVM/runtimes. In this path
> the build system will automatically build with the just-built compiler.
> This path also splits compiler-rt into two separate build steps, one that
> configures and builds the builtins with the just-built compiler, and a
> second that configures and builds the sanitizer libraries.
>
>
> Huh, OK - could someone remove the legacy format, then? If it's a trap.
>
>
> I'm not sure the new path is fully supported in every workflow, so
> removing it seems like a not great idea at the moment.
>
>
> That said, I tried putting compiler-rt in runtimes instead of projects and
> I got a bunch of cmake errors starting with:
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "GotsanRuntimeCheck" of target
"check-runtimes"
> does
>   not exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
>
> Any ideas?
>
>
> I have never encountered that issue. It looks like the tsan test code is
> out of sync. If you go into tsan/test/CMakeLists.txt and on Line 2 add this
> to the if statement "AND TARGET GotsanRuntimeCheck" that should
fix the
> issue.
>
>
> Hrm - not sure which CMakeLists.txt you're referring to? In my
> runtimes/compiler-rt/lib/tsan/tests/CMakeLists.txt the first few lines are:
>
>
> Sorry, I meant <compiler-rt>/test/tsan/CMakeLists.txt, not tsan/test.
>
Think that got me past that error, but (sorry I wasn't clear) it was only
one of many errors. Here are the next... 5, say...

CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
  The dependency target "TsanUnitTests" of target
"check-runtimes" does not
  exist.
Call Stack (most recent call first):
  CMakeLists.txt:110 (add_lit_target)


CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
  The dependency target "cfi" of target "check-runtimes"
does not exist.
Call Stack (most recent call first):
  CMakeLists.txt:110 (add_lit_target)


CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
  The dependency target "TsanUnitTests" of target
"check-compiler-rt" does
  not exist.
Call Stack (most recent call first):
  compiler-rt/test/CMakeLists.txt:94 (add_lit_target)


CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
  The dependency target "cfi" of target "check-compiler-rt"
does not exist.
Call Stack (most recent call first):
  compiler-rt/test/CMakeLists.txt:94 (add_lit_target)


CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
  The dependency target "TsanUnitTests" of target
"check-tsan" does not
  exist.
Call Stack (most recent call first):

/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1195
(add_lit_target)
  compiler-rt/test/tsan/CMakeLists.txt:46 (add_lit_testsuite)


I'm assuming there's some systemic problem with the way I'm holding
this
(if it's working for other people)?

- Dave

>
> -Chris
>
>
> include_directories(../rtl)
>
> add_custom_target(TsanUnitTests)
> set_target_properties(TsanUnitTests PROPERTIES
>   FOLDER "TSan unittests")
>
> no if condition I could modify?
>
>
>
> -Chris
>
>
>
>
> The second path also works for many (but not all) of our other runtime
> library projects. I know it works for libcxx, libcxxabi, and
> libunwind. Petr Hosek (CC'd) has also been working on support for
> multi-arch builtin and runtime library builds so that you can generate full
> cross-compilers from a single cmake invocation.
>
> -Chris
>
>
> On Mar 8, 2017, at 2:35 PM, David Blaikie via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
>
> On Wed, Mar 8, 2017 at 2:03 PM Sterling Augustine <saugustine at
google.com>
> wrote:
>
> Yes, this is a aspect of the larger problem that clang bootstrap
doesn't
> work for a cross-compiler. The build (mostly?) assumes that host==target
> during the build of clang itself, and then if you want another architecture
> also, you run a second build of the target libraries, and manually merge
> the trees.
>
>
> I kind of roughly follow that, but not too well.
>
>
> If you think about compiler-rt as being compiled for the target rather
> than the host, the problem you describe here is exactly the same one, and
> we have been getting lucky.
>
>
> Sure - if a PPC clang is being built from an x86 host, how would
> compiler-rt be built (OK, it could be built with the just-built clang,
> which it isn't at the moment) and tested (can't really be tested
because
> the host can't run PPC binaries).
>
>
> At the moment, the blaze builds of clang do exactly the procedure
> described above, so this hasn't been a terrible problem for Google, but
I
> do think it is something that should be fixed (I'm working on another
> aspect of compiler-rt bringup at the moment, so won't solve this in the
> immediate future.)
>
>
> Rightio
>
>
>
> gnu systems have a make variable, "CC_FOR_TARGET" that addresses
this
> problem. I imagine llvm should adopt a similar mechanism inside cmake.
>
>
> Not sure I follow on the need/use of CC_FOR_TARGET compared to using the
> just-built clang as the CC_FOR_TARGET (which it seems we have some plumbing
> for already - the just-built clang is used for building the compiler-rt
> tests, but not for building the library. I /think/ it should be used for
> both)
>
> - Dave
>
>
>
> On Wed, Mar 8, 2017 at 1:54 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
> I stumbled across what seems to be a bug (to me) in the compiler-rt build:
>
> The compiler-rt libraries themselves are built with the host compiler
> while the tests are built and then linked with the just-built clang.
>
> It was my understanding that the goal/intent/need was to have the
> compiler-rt library build with the just-built clang? Did I misunderstand
> that?*
>
> Sterling: Chandler seemed to think you might be interested in this issue
&
> possibly addressing it given you're working on compiler-rt bring-up?
It'd
> probably be useful to have compiler-rt built with the just-built clang for
> performance reasons.
>
> Evgeniy - not sure if you're interested in this or have much context?
Know
> if this is right/wrong/neutral, etc?
>
> * reasons include performance, ABI compatibility, etc (I thought this was
> necessary for correctness in some way) - also, otherwise it seems excessive
> to hold up the whole build on waiting for just-built clang to finish, then
> use that to compile some tests. (well, I realize some of the tests are
> end-to-end, so they do need the just-built compiler)
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170309/d6adc12c/attachment.html>

I'll try and reproduce later today. Is this Linux? Can you give me your
CMake command line?

-Chris
> On Mar 9, 2017, at 11:45 AM, David Blaikie <dblaikie at gmail.com>
wrote:
> 
> 
> On Thu, Mar 9, 2017 at 11:25 AM Chris Bieneman <beanz at apple.com
<mailto:beanz at apple.com>> wrote:
>> On Mar 8, 2017, at 4:42 PM, David Blaikie <dblaikie at gmail.com
<mailto:dblaikie at gmail.com>> wrote:
>> 
>> 
>> 
>> On Wed, Mar 8, 2017 at 3:23 PM Chris Bieneman <beanz at apple.com
<mailto:beanz at apple.com>> wrote:
>>> On Mar 8, 2017, at 3:16 PM, David Blaikie <dblaikie at gmail.com
<mailto:dblaikie at gmail.com>> wrote:
>>> 
>>> 
>>> 
>>> On Wed, Mar 8, 2017 at 2:55 PM Chris Bieneman <beanz at
apple.com <mailto:beanz at apple.com>> wrote:
>>> David,
>>> 
>>> This is an area that has had a lot of development over the last two
years.
>>> 
>>> There are two supported ways in the LLVM build system to build
compiler-rt with the just-built compiler.
>>> 
>>> 1) The legacy way is for if compiler-rt is under LLVM/projects. You
can specify -DLLVM_BUILD_EXTERNAL_COMPILER_RT=On, which will configure
compiler-rt using the just-built clang after clang is built.
>>> 
>>> I thought the BUILD_EXTERNAL variables were for use when projects
were not embedded within the llvm source tree (mostly in use by Takumi's
flat buildbots that checout the top-level project without embedding, say, clang
or compiler-rt within the llvm source tree)?
>> 
>> You are confusing this with the similarly named
LLVM_EXTERNAL_${nameUPPER}_SOURCE_DIR variables.
>> 
>> Ah, right - indeed.
>>> 
>>> 2) The new way, is to place compiler-rt under LLVM/runtimes. In
this path the build system will automatically build with the just-built
compiler. This path also splits compiler-rt into two separate build steps, one
that configures and builds the builtins with the just-built compiler, and a
second that configures and builds the sanitizer libraries.
>>> 
>>> Huh, OK - could someone remove the legacy format, then? If it's
a trap.
>> 
>> I'm not sure the new path is fully supported in every workflow, so
removing it seems like a not great idea at the moment.
>> 
>>> 
>>> That said, I tried putting compiler-rt in runtimes instead of
projects and I got a bunch of cmake errors starting with:
>>> 
>>> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>>>   The dependency target "GotsanRuntimeCheck" of target
"check-runtimes" does
>>>   not exist.
>>> Call Stack (most recent call first):
>>>   CMakeLists.txt:110 (add_lit_target)
>>> 
>>> Any ideas?
>> 
>> I have never encountered that issue. It looks like the tsan test code
is out of sync. If you go into tsan/test/CMakeLists.txt and on Line 2 add this
to the if statement "AND TARGET GotsanRuntimeCheck" that should fix
the issue.
>> 
>> Hrm - not sure which CMakeLists.txt you're referring to? In my
runtimes/compiler-rt/lib/tsan/tests/CMakeLists.txt the first few lines are:
> 
> Sorry, I meant <compiler-rt>/test/tsan/CMakeLists.txt, not tsan/test.
> 
> Think that got me past that error, but (sorry I wasn't clear) it was
only one of many errors. Here are the next... 5, say...
> 
> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-runtimes" does not
>   exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
> 
> 
> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>   The dependency target "cfi" of target
"check-runtimes" does not exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
> 
> 
> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-compiler-rt" does
>   not exist.
> Call Stack (most recent call first):
>   compiler-rt/test/CMakeLists.txt:94 (add_lit_target)
> 
> 
> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>   The dependency target "cfi" of target
"check-compiler-rt" does not exist.
> Call Stack (most recent call first):
>   compiler-rt/test/CMakeLists.txt:94 (add_lit_target)
> 
> 
> CMake Error at
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
(add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-tsan" does not
>   exist.
> Call Stack (most recent call first):
>  
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1195
(add_lit_target)
>   compiler-rt/test/tsan/CMakeLists.txt:46 (add_lit_testsuite)
> 
> 
> I'm assuming there's some systemic problem with the way I'm
holding this (if it's working for other people)?
> 
> - Dave
>  
> 
> -Chris
> 
>> 
>> include_directories(../rtl)
>> 
>> add_custom_target(TsanUnitTests)
>> set_target_properties(TsanUnitTests PROPERTIES
>>   FOLDER "TSan unittests")
>> 
>> no if condition I could modify?
>>  
>> 
>> -Chris
>> 
>>>  
>>> 
>>> The second path also works for many (but not all) of our other
runtime library projects. I know it works for libcxx, libcxxabi, and libunwind.
Petr Hosek (CC'd) has also been working on support for multi-arch builtin
and runtime library builds so that you can generate full cross-compilers from a
single cmake invocation.
>>> 
>>> -Chris
>>> 
>>> 
>>> 
>>>> On Mar 8, 2017, at 2:35 PM, David Blaikie via llvm-dev
<llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
wrote:
>>>> 
>>> 
>>>> 
>>>> 
>>>> On Wed, Mar 8, 2017 at 2:03 PM Sterling Augustine
<saugustine at google.com <mailto:saugustine at google.com>> wrote:
>>>> Yes, this is a aspect of the larger problem that clang
bootstrap doesn't work for a cross-compiler. The build (mostly?) assumes
that host==target during the build of clang itself, and then if you want another
architecture also, you run a second build of the target libraries, and manually
merge the trees.
>>>> 
>>>> I kind of roughly follow that, but not too well.
>>>>  
>>>> If you think about compiler-rt as being compiled for the target
rather than the host, the problem you describe here is exactly the same one, and
we have been getting lucky.
>>>> 
>>>> Sure - if a PPC clang is being built from an x86 host, how
would compiler-rt be built (OK, it could be built with the just-built clang,
which it isn't at the moment) and tested (can't really be tested because
the host can't run PPC binaries).
>>>>  
>>>> At the moment, the blaze builds of clang do exactly the
procedure described above, so this hasn't been a terrible problem for
Google, but I do think it is something that should be fixed (I'm working on
another aspect of compiler-rt bringup at the moment, so won't solve this in
the immediate future.)
>>>> 
>>>> Rightio
>>>>  
>>>> 
>>>> gnu systems have a make variable, "CC_FOR_TARGET"
that addresses this problem. I imagine llvm should adopt a similar mechanism
inside cmake.
>>>> 
>>>> Not sure I follow on the need/use of CC_FOR_TARGET compared to
using the just-built clang as the CC_FOR_TARGET (which it seems we have some
plumbing for already - the just-built clang is used for building the compiler-rt
tests, but not for building the library. I /think/ it should be used for both)
>>>> 
>>>> - Dave
>>>>  
>>>> 
>>>> On Wed, Mar 8, 2017 at 1:54 PM, David Blaikie <dblaikie at
gmail.com <mailto:dblaikie at gmail.com>> wrote:
>>>> I stumbled across what seems to be a bug (to me) in the
compiler-rt build:
>>>> 
>>>> The compiler-rt libraries themselves are built with the host
compiler while the tests are built and then linked with the just-built clang.
>>>> 
>>>> It was my understanding that the goal/intent/need was to have
the compiler-rt library build with the just-built clang? Did I misunderstand
that?*
>>>> 
>>>> Sterling: Chandler seemed to think you might be interested in
this issue & possibly addressing it given you're working on compiler-rt
bring-up? It'd probably be useful to have compiler-rt built with the
just-built clang for performance reasons.
>>>> 
>>>> Evgeniy - not sure if you're interested in this or have
much context? Know if this is right/wrong/neutral, etc?
>>>> 
>>>> * reasons include performance, ABI compatibility, etc (I
thought this was necessary for correctness in some way) - also, otherwise it
seems excessive to hold up the whole build on waiting for just-built clang to
finish, then use that to compile some tests. (well, I realize some of the tests
are end-to-end, so they do need the just-built compiler)
>>>> 
>>> 
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
<http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170309/38d43d3e/attachment.html>

On Thu, Mar 9, 2017 at 3:00 PM Chris Bieneman <beanz at apple.com> wrote:
> I'll try and reproduce later today. Is this Linux? Can you give me your
> CMake command line?
>
Excuse the delay, been busy setting up a new machine - also an opportunity
to try clean cmake setups rather than my aging configurations that have a
bunch of old stuff baked in and manual variables changed, etc. (& I do so
wish that CMakeCache.txt had a comment at teh top describing the command
that produced it like the config.log - that way I'd probably be more likely
to copy/paste and modify the command line than hand editing the cache file)

Anyway, so I don't have the compounding issue (local libstdc++ install with
ABI incompatibility to my system libstdc++) so I can build/test compiler-rt
when it's in projects at the moment (though I will soon locally install GCC
6.3 and have the problems with xray again).

But when I place compiler-rt in runtimes I do have problems.

Here's my cmake command:

cmake -G Ninja -DCMAKE_INSTALL_PREFIX=$HOME/dev/llvm/install
-DCMAKE_BUILD_TYPE=Debug ../../src/
-DCMAKE_C_COMPILER=$HOME/install/bin/clang
-DCMAKE_CXX_COMPILER=$HOME/install/bin/clang++ -DLLVM_USE_SPLIT_DWARF=ON
-DLLVM_ENABLE_WERROR=ON -DLLVM_BUILD_EXAMPLES=ON -DLLVM_INCLUDE_GO_TESTS=OFF

& here's the result of "ninja check-all":

[3671/3683] cd
/usr/local/google/home/blaikie/dev/llvm/build/default...fault/runtimes/runtimes-bins/
--target check-runtimes --config Debug
[318/409] Generating tsan_mman_test.cc.x86_64.o
FAILED: compiler-rt/lib/tsan/tests/unit/tsan_mman_test.cc.x86_64.o
cd
/usr/local/google/home/blaikie/dev/llvm/build/default/runtimes/runtimes-bins/compiler-rt/lib/tsan/tests/unit
&& /usr/local/google/home/blaikie/dev/llvm/build/default/./bin/clang
-fPIC
-fvisibility-inlines-hidden -Werror -Werror=date-time -std=c++11
-fcolor-diagnostics -Wall -Werror -std=c++11 -Wno-unused-parameter
-Wno-unknown-warning-option -fPIC -fno-builtin -fno-exceptions
-fomit-frame-pointer -funwind-tables -fno-stack-protector
-fno-sanitize=safe-stack -fvisibility=hidden -fvisibility-inlines-hidden
-fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros
-Wno-c99-extensions -Wno-non-virtual-dtor -fPIE -fno-rtti
-Wno-covered-switch-default -DGTEST_NO_LLVM_RAW_OSTREAM=1
-DGTEST_HAS_RTTI=0
-I/usr/local/google/home/blaikie/dev/llvm/src/utils/unittest/googletest/include
-I/usr/local/google/home/blaikie/dev/llvm/src/utils/unittest/googletest
-I/usr/local/google/home/blaikie/dev/llvm/src/runtimes/compiler-rt/lib
-I/usr/local/google/home/blaikie/dev/llvm/src/runtimes/compiler-rt/lib/tsan/rtl
-DGTEST_HAS_RTTI=0 -m64 -c -o tsan_mman_test.cc.x86_64.o
/usr/local/google/home/blaikie/dev/llvm/src/runtimes/compiler-rt/lib/tsan/tests/unit/tsan_mman_test.cc
/usr/local/google/home/blaikie/dev/llvm/src/runtimes/compiler-rt/lib/tsan/tests/unit/tsan_mman_test.cc:14:10:
fatal error: 'sanitizer/allocator_interface.h' file not found
#include <sanitizer/allocator_interface.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
[370/409] Generating ASAN_INST_TEST_OBJECTS.asan_test.cc.x86_64-inline.o

> -Chris
>
> On Mar 9, 2017, at 11:45 AM, David Blaikie <dblaikie at gmail.com>
wrote:
>
>
> On Thu, Mar 9, 2017 at 11:25 AM Chris Bieneman <beanz at apple.com>
wrote:
>
> On Mar 8, 2017, at 4:42 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
>
>
> On Wed, Mar 8, 2017 at 3:23 PM Chris Bieneman <beanz at apple.com>
wrote:
>
> On Mar 8, 2017, at 3:16 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
>
>
> On Wed, Mar 8, 2017 at 2:55 PM Chris Bieneman <beanz at apple.com>
wrote:
>
> David,
>
> This is an area that has had a lot of development over the last two years.
>
> There are two supported ways in the LLVM build system to build compiler-rt
> with the just-built compiler.
>
> 1) The legacy way is for if compiler-rt is under LLVM/projects. You can
> specify -DLLVM_BUILD_EXTERNAL_COMPILER_RT=On, which will configure
> compiler-rt using the just-built clang after clang is built.
>
>
> I thought the BUILD_EXTERNAL variables were for use when projects were not
> embedded within the llvm source tree (mostly in use by Takumi's flat
> buildbots that checout the top-level project without embedding, say, clang
> or compiler-rt within the llvm source tree)?
>
>
> You are confusing this with the similarly
> named LLVM_EXTERNAL_${nameUPPER}_SOURCE_DIR variables.
>
>
> Ah, right - indeed.
>
>
> 2) The new way, is to place compiler-rt under LLVM/runtimes. In this path
> the build system will automatically build with the just-built compiler.
> This path also splits compiler-rt into two separate build steps, one that
> configures and builds the builtins with the just-built compiler, and a
> second that configures and builds the sanitizer libraries.
>
>
> Huh, OK - could someone remove the legacy format, then? If it's a trap.
>
>
> I'm not sure the new path is fully supported in every workflow, so
> removing it seems like a not great idea at the moment.
>
>
> That said, I tried putting compiler-rt in runtimes instead of projects and
> I got a bunch of cmake errors starting with:
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "GotsanRuntimeCheck" of target
"check-runtimes"
> does
>   not exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
>
> Any ideas?
>
>
> I have never encountered that issue. It looks like the tsan test code is
> out of sync. If you go into tsan/test/CMakeLists.txt and on Line 2 add this
> to the if statement "AND TARGET GotsanRuntimeCheck" that should
fix the
> issue.
>
>
> Hrm - not sure which CMakeLists.txt you're referring to? In my
> runtimes/compiler-rt/lib/tsan/tests/CMakeLists.txt the first few lines are:
>
>
> Sorry, I meant <compiler-rt>/test/tsan/CMakeLists.txt, not tsan/test.
>
>
> Think that got me past that error, but (sorry I wasn't clear) it was
only
> one of many errors. Here are the next... 5, say...
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-runtimes" does not
>   exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
>
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "cfi" of target
"check-runtimes" does not exist.
> Call Stack (most recent call first):
>   CMakeLists.txt:110 (add_lit_target)
>
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-compiler-rt" does
>   not exist.
> Call Stack (most recent call first):
>   compiler-rt/test/CMakeLists.txt:94 (add_lit_target)
>
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "cfi" of target
"check-compiler-rt" does not exist.
> Call Stack (most recent call first):
>   compiler-rt/test/CMakeLists.txt:94 (add_lit_target)
>
>
> CMake Error at
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1174
> (add_dependencies):
>   The dependency target "TsanUnitTests" of target
"check-tsan" does not
>   exist.
> Call Stack (most recent call first):
>
>
/usr/local/google/home/blaikie/dev/llvm/build/clang/debug/split/notypes/nostandalone/lib/cmake/llvm/AddLLVM.cmake:1195
> (add_lit_target)
>   compiler-rt/test/tsan/CMakeLists.txt:46 (add_lit_testsuite)
>
>
> I'm assuming there's some systemic problem with the way I'm
holding this
> (if it's working for other people)?
>
> - Dave
>
>
>
> -Chris
>
>
> include_directories(../rtl)
>
> add_custom_target(TsanUnitTests)
> set_target_properties(TsanUnitTests PROPERTIES
>   FOLDER "TSan unittests")
>
> no if condition I could modify?
>
>
>
> -Chris
>
>
>
>
> The second path also works for many (but not all) of our other runtime
> library projects. I know it works for libcxx, libcxxabi, and
> libunwind. Petr Hosek (CC'd) has also been working on support for
> multi-arch builtin and runtime library builds so that you can generate full
> cross-compilers from a single cmake invocation.
>
> -Chris
>
>
> On Mar 8, 2017, at 2:35 PM, David Blaikie via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
>
> On Wed, Mar 8, 2017 at 2:03 PM Sterling Augustine <saugustine at
google.com>
> wrote:
>
> Yes, this is a aspect of the larger problem that clang bootstrap
doesn't
> work for a cross-compiler. The build (mostly?) assumes that host==target
> during the build of clang itself, and then if you want another architecture
> also, you run a second build of the target libraries, and manually merge
> the trees.
>
>
> I kind of roughly follow that, but not too well.
>
>
> If you think about compiler-rt as being compiled for the target rather
> than the host, the problem you describe here is exactly the same one, and
> we have been getting lucky.
>
>
> Sure - if a PPC clang is being built from an x86 host, how would
> compiler-rt be built (OK, it could be built with the just-built clang,
> which it isn't at the moment) and tested (can't really be tested
because
> the host can't run PPC binaries).
>
>
> At the moment, the blaze builds of clang do exactly the procedure
> described above, so this hasn't been a terrible problem for Google, but
I
> do think it is something that should be fixed (I'm working on another
> aspect of compiler-rt bringup at the moment, so won't solve this in the
> immediate future.)
>
>
> Rightio
>
>
>
> gnu systems have a make variable, "CC_FOR_TARGET" that addresses
this
> problem. I imagine llvm should adopt a similar mechanism inside cmake.
>
>
> Not sure I follow on the need/use of CC_FOR_TARGET compared to using the
> just-built clang as the CC_FOR_TARGET (which it seems we have some plumbing
> for already - the just-built clang is used for building the compiler-rt
> tests, but not for building the library. I /think/ it should be used for
> both)
>
> - Dave
>
>
>
> On Wed, Mar 8, 2017 at 1:54 PM, David Blaikie <dblaikie at gmail.com>
wrote:
>
> I stumbled across what seems to be a bug (to me) in the compiler-rt build:
>
> The compiler-rt libraries themselves are built with the host compiler
> while the tests are built and then linked with the just-built clang.
>
> It was my understanding that the goal/intent/need was to have the
> compiler-rt library build with the just-built clang? Did I misunderstand
> that?*
>
> Sterling: Chandler seemed to think you might be interested in this issue
&
> possibly addressing it given you're working on compiler-rt bring-up?
It'd
> probably be useful to have compiler-rt built with the just-built clang for
> performance reasons.
>
> Evgeniy - not sure if you're interested in this or have much context?
Know
> if this is right/wrong/neutral, etc?
>
> * reasons include performance, ABI compatibility, etc (I thought this was
> necessary for correctness in some way) - also, otherwise it seems excessive
> to hold up the whole build on waiting for just-built clang to finish, then
> use that to compile some tests. (well, I realize some of the tests are
> end-to-end, so they do need the just-built compiler)
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170311/f5b01fb8/attachment.html>