thr3ads.net - llvm dev - [llvm-dev] Cmake-gen'd parallel make breaks on native tablegen [Oct 2015]

If this information is useful, please help other people find it:
Share via:

Chris Bieneman via llvm-dev

2015-Oct-07 23:18 UTC

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

It should probably be inside an `if(LLVM_USE_HOST_TOOLS)` block.

That way the extra target and command won’t get added unless needed and CMake
won’t spew dev warnings. Adding a target with a nonexistent dependency makes
CMake dump a bunch of developer warnings.

Other than that this looks good. Does it resolve the issue for you?

-Chris
> On Oct 7, 2015, at 4:09 PM, Alex Wang <aw1621107 at gmail.com> wrote:
> 
> diff --git a/cmake/modules/TableGen.cmake b/cmake/modules/TableGen.cmake
> index 452a728..b24197b 100644
> --- a/cmake/modules/TableGen.cmake
> +++ b/cmake/modules/TableGen.cmake
> @@ -70,6 +70,13 @@ function(add_public_tablegen_target target)
>   set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} ${target} PARENT_SCOPE)
> endfunction()
> 
> +add_custom_command(OUTPUT LIB_LLVMSUPPORT
> +    COMMAND ${CMAKE_COMMAND} --build . --target LLVMSupport --config
Release
> +    DEPENDS CONFIGURE_LLVM_NATIVE
> +    WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
> +    COMMENT "Building libLLVMSupport for native TableGen...")
> +add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS LIB_LLVMSUPPORT)
> +
> macro(add_tablegen target project)
>   set(${target}_OLD_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
>   set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
> @@ -109,7 +116,7 @@ macro(add_tablegen target project)
> 
>       add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>         COMMAND ${CMAKE_COMMAND} --build . --target ${target} --config
Release
> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>         WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>         COMMENT "Building native TableGen...")
>       add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
> 
> Something like this?
> 
>> On Oct 6, 2015, at 7:58 PM, Chris Bieneman <beanz at apple.com>
wrote:
>> 
>> 
>>> On Oct 5, 2015, at 9:37 AM, Alex Wang via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>> 
>>> Probably should have checked earlier... Do patches go directly to
llvm-commits for review instead of getting reviewed here first?
>>> 
>>> ccing llvm-commits too in case
>>>> On Oct 4, 2015, at 8:20 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>>> 
>>>> Alright, got something thrown together. Here it is inline (also
attached if that's more convenient):
>>>> 
>>>> diff --git a/cmake/modules/TableGen.cmake
b/cmake/modules/TableGen.cmake
>>>> index 452a728..be6729d 100644
>>>> --- a/cmake/modules/TableGen.cmake
>>>> +++ b/cmake/modules/TableGen.cmake
>>>> @@ -107,9 +107,18 @@ macro(add_tablegen target project)
>>>>    endif()
>>>>    set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN_EXE}
PARENT_SCOPE)
>>>> 
>>>> +      if(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>> 
>> Instead of doing this you could move creating this out of the macro.
Then it will be executed when the file is included which should only be once.
>> 
>>>> +        add_custom_command(OUTPUT LIB_LLVMSUPPORT
>>>> +          COMMAND ${CMAKE_COMMAND} --build . --target
LLVMSupport --config Release
>>>> +          DEPENDS CONFIGURE_LLVM_NATIVE
>>>> +          WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>> +          COMMENT "Building libLLVMSupport for native
TableGen...")
>>>> +        add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS
LIB_LLVMSUPPORT)
>>>> +      endif(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>>>> +
>>>>    add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>>>>      COMMAND ${CMAKE_COMMAND} --build . --target ${target}
--config Release
>>>> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>>>> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>>>>      WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>      COMMENT "Building native TableGen...")
>>>>    add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
>>>> 
>>>> Basically does exactly what Chris suggested: Make a custom
command + target just for the native libLLVMSupport, and have the existing stuff
depend on that.
>>>> 
>>>> I don't really like having to check to see if
NATIVE_LIB_LLVMSUPPORT is already defined as a target, but I'm not sure what
else to do. It looks like this particular macro is called multiple times, and I
don't think defining this elsewhere wouldn't be very good because
it's only used here. It shouldn't have a significant effect, though,
because the cmake prep is so short compared to the actual build.
>>>> 
>>>> -Alex
>>>> 
>>>> <TableGen.cmake.patch>
>>>>> On Sep 28, 2015, at 5:19 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>>>> 
>>>>> That sounds much more reasonable than more directories.
I'll see if I can't hack something together...
>>>>> 
>>>>> I still think you have a point; I would have expected cmake
to catch that dependency. Strange thing is that just that library had an
independent cmake build going, with different percents showing and not being
verbose when the main config had it on.
>>>>> 
>>>>> -Alex
>>>>> 
>>>>>> On Sep 28, 2015, at 5:02 PM, Chris Bieneman <beanz
at apple.com> wrote:
>>>>>> 
>>>>>> I think I understand what you think the issue is. I
would have assumed that the generated makefiles could handle two targets
building simultaneously, but you may be right.
>>>>>> 
>>>>>> I don’t think we should solve this with separate
directories though because that is a waste. A better solution would be making a
custom command & target that builds the native libSupport, then make all
tablegen targets depend on that one.
>>>>>> 
>>>>>> That way we only build the native libSupport once.
>>>>>> 
>>>>>> -Chris
>>>>>> 
>>>>>>> On Sep 28, 2015, at 1:48 PM, Alex Wang
<aw1621107 at gmail.com> wrote:
>>>>>>> 
>>>>>>>> 
>>>>>>>> On Sep 28, 2015, at 3:56 PM, Chris Bieneman
<beanz at apple.com> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Sep 28, 2015, at 12:07 PM, Alex Wang via
llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>>>> 
>>>>>>>>> Hello backend devs,
>>>>>>>>> 
>>>>>>>>> Been working on a parallelization bug in
the cmake configs, where parallel makefile builds of llvm with clang and native
tablegen would usually break during linking in either either the clang_tblgen or
llvm_tblgen targets. Looking at the cmake command that generates the tablegen
makefile commands (I think) (cmake/modules/TableGen.cmake:113-117):
>>>>>>>>> 
>>>>>>>>> add_custom_command(OUTPUT
${${project}_TABLEGEN_EXE}
>>>>>>>>>   COMMAND ${CMAKE_COMMAND} --build ../..
--target ${target} --config Release
>>>>>>>>>   DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>>>>>>>>>   WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>>>>>>   COMMENT "Building native
TableGen...")
>>>>>>>>> 
>>>>>>>>> My guess is that the clang and llvm
tablegen builds are trying to place their output into the same folder, so
depending on when the resulting library gets linked/moved one of the threads
could be left dangling with a partially-completed library or no library.
>>>>>>>> 
>>>>>>>> If you build with LLVM_OPTIMIZED_TABLEGEN the
native tablegen is a fully generated build subtree. Make seems to have some
problems with thread scheduling with these builds. I haven’t diagnosed the
issue, but it isn’t that the builds are writing to the same directory. It
happens regardless of whether you’re building llvm with or without clang, and
the llvm & clang tablegen executables are named differently so they don’t
conflict.
>>>>>>> 
>>>>>>> Hmmm, that's odd. I never saw the build break
like this when building only llvm. Seems I just got really (un)lucky...
>>>>>>> 
>>>>>>> Probably could have explained this better... It
wasn't that I thought the output executables would clash; I thought that
both the llvm and clang tablegen builds were working on libLLVMSupport at the
same time in the same place, so when one finished and moved the library to its
proper place the other thread is left without anything to work with. Seems like
what you said would still apply though.
>>>>>>> 
>>>>>>> Another thing to note is that when building with
clang the cmake percentages for that particular library would always go above
100%. I thought that meant that they could be writing to the same progress file,
because that never happened with llvm-only builds.
>>>>>>> 
>>>>>>>>> The error was either "Couldn't
stat output file: ../LLVMSupport.a No such file or directory" or something
about an archive member extending past the end of the file, and the cmake
progress messages before linking were printed twice. Once for each thread,
I'd guess? Sort of surprised that they seem to share so much code (or at
least filenames)
>>>>>>>>> 
>>>>>>>>> First thing I could think of to fix this is
adding another layer or two of CMakeLists, so the tablegen creation command
would be invoked in different folders. I'm not convinced that that is the
best fix, and I couldn't figure out how to move the builds into different
folders with only minor changes to the existing cmake files.
>>>>>>>> 
>>>>>>>> I don’t think that will work. As I said above,
they are building to different directories. You can note that in the custom
command setting WORKING_DIRECTORY to ${LLVM_NATIVE_BUILD}, which is a nested
fully-independent cmake build tree.
>>>>>>>> 
>>>>>>> 
>>>>>>> At least on my machine, ${LLVM_NATIVE_BUILD} is the
same for both llvm and clang tablegen commands.
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> Thoughts?
>>>>>>>>> 
>>>>>>>>> -Alex
>>>>>>>>> 
>>>>>>>>> (CrossCompile.cmake also has a typo in the
first line: "toochain" --> "toolchain", if that is
something to worry about)
>>>>>>>>>
_______________________________________________
>>>>>>>>> LLVM Developers mailing list
>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>> 
>>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> 
>

Alex Wang via llvm-dev

2015-Oct-08 00:15 UTC

head link

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

diff --git a/cmake/modules/TableGen.cmake b/cmake/modules/TableGen.cmake
index 452a728..cb06450 100644
--- a/cmake/modules/TableGen.cmake
+++ b/cmake/modules/TableGen.cmake
@@ -70,6 +70,15 @@ function(add_public_tablegen_target target)
   set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} ${target} PARENT_SCOPE)
 endfunction()

+if(LLVM_USE_HOST_TOOLS)
+  add_custom_command(OUTPUT LIB_LLVMSUPPORT
+      COMMAND ${CMAKE_COMMAND} --build . --target LLVMSupport --config Release
+      DEPENDS CONFIGURE_LLVM_NATIVE
+      WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
+      COMMENT "Building libLLVMSupport for native TableGen...")
+  add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS LIB_LLVMSUPPORT)
+endif(LLVM_USE_HOST_TOOLS)
+
 macro(add_tablegen target project)
   set(${target}_OLD_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
   set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
@@ -109,7 +118,7 @@ macro(add_tablegen target project)

       add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
         COMMAND ${CMAKE_COMMAND} --build . --target ${target} --config Release
-        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
+        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
         WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
         COMMENT "Building native TableGen...")
       add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})

The first version definitely worked, but I can't confirm that this version
works because I'm getting compiler segfaults in Apple's clang when
trying to compile... Can't take a close look right now, but I definitely
will sometime tonight.

-Alex
> On Oct 7, 2015, at 7:18 PM, Chris Bieneman <beanz at apple.com>
wrote:
> 
> It should probably be inside an `if(LLVM_USE_HOST_TOOLS)` block.
> 
> That way the extra target and command won’t get added unless needed and
CMake won’t spew dev warnings. Adding a target with a nonexistent dependency
makes CMake dump a bunch of developer warnings.
> 
> Other than that this looks good. Does it resolve the issue for you?
> 
> -Chris
> 
>> On Oct 7, 2015, at 4:09 PM, Alex Wang <aw1621107 at gmail.com>
wrote:
>> 
>> diff --git a/cmake/modules/TableGen.cmake
b/cmake/modules/TableGen.cmake
>> index 452a728..b24197b 100644
>> --- a/cmake/modules/TableGen.cmake
>> +++ b/cmake/modules/TableGen.cmake
>> @@ -70,6 +70,13 @@ function(add_public_tablegen_target target)
>>  set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} ${target} PARENT_SCOPE)
>> endfunction()
>> 
>> +add_custom_command(OUTPUT LIB_LLVMSUPPORT
>> +    COMMAND ${CMAKE_COMMAND} --build . --target LLVMSupport --config
Release
>> +    DEPENDS CONFIGURE_LLVM_NATIVE
>> +    WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>> +    COMMENT "Building libLLVMSupport for native
TableGen...")
>> +add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS LIB_LLVMSUPPORT)
>> +
>> macro(add_tablegen target project)
>>  set(${target}_OLD_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
>>  set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
>> @@ -109,7 +116,7 @@ macro(add_tablegen target project)
>> 
>>      add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>>        COMMAND ${CMAKE_COMMAND} --build . --target ${target} --config
Release
>> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>>        WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>        COMMENT "Building native TableGen...")
>>      add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
>> 
>> Something like this?
>> 
>>> On Oct 6, 2015, at 7:58 PM, Chris Bieneman <beanz at
apple.com> wrote:
>>> 
>>> 
>>>> On Oct 5, 2015, at 9:37 AM, Alex Wang via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>>> 
>>>> Probably should have checked earlier... Do patches go directly
to llvm-commits for review instead of getting reviewed here first?
>>>> 
>>>> ccing llvm-commits too in case
>>>>> On Oct 4, 2015, at 8:20 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>>>> 
>>>>> Alright, got something thrown together. Here it is inline
(also attached if that's more convenient):
>>>>> 
>>>>> diff --git a/cmake/modules/TableGen.cmake
b/cmake/modules/TableGen.cmake
>>>>> index 452a728..be6729d 100644
>>>>> --- a/cmake/modules/TableGen.cmake
>>>>> +++ b/cmake/modules/TableGen.cmake
>>>>> @@ -107,9 +107,18 @@ macro(add_tablegen target project)
>>>>>   endif()
>>>>>   set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN_EXE}
PARENT_SCOPE)
>>>>> 
>>>>> +      if(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>>> 
>>> Instead of doing this you could move creating this out of the
macro. Then it will be executed when the file is included which should only be
once.
>>> 
>>>>> +        add_custom_command(OUTPUT LIB_LLVMSUPPORT
>>>>> +          COMMAND ${CMAKE_COMMAND} --build . --target
LLVMSupport --config Release
>>>>> +          DEPENDS CONFIGURE_LLVM_NATIVE
>>>>> +          WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>> +          COMMENT "Building libLLVMSupport for native
TableGen...")
>>>>> +        add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS
LIB_LLVMSUPPORT)
>>>>> +      endif(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>>>>> +
>>>>>   add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>>>>>     COMMAND ${CMAKE_COMMAND} --build . --target ${target}
--config Release
>>>>> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>>>>> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>>>>>     WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>>     COMMENT "Building native TableGen...")
>>>>>   add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
>>>>> 
>>>>> Basically does exactly what Chris suggested: Make a custom
command + target just for the native libLLVMSupport, and have the existing stuff
depend on that.
>>>>> 
>>>>> I don't really like having to check to see if
NATIVE_LIB_LLVMSUPPORT is already defined as a target, but I'm not sure what
else to do. It looks like this particular macro is called multiple times, and I
don't think defining this elsewhere wouldn't be very good because
it's only used here. It shouldn't have a significant effect, though,
because the cmake prep is so short compared to the actual build.
>>>>> 
>>>>> -Alex
>>>>> 
>>>>> <TableGen.cmake.patch>
>>>>>> On Sep 28, 2015, at 5:19 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>>>>> 
>>>>>> That sounds much more reasonable than more directories.
I'll see if I can't hack something together...
>>>>>> 
>>>>>> I still think you have a point; I would have expected
cmake to catch that dependency. Strange thing is that just that library had an
independent cmake build going, with different percents showing and not being
verbose when the main config had it on.
>>>>>> 
>>>>>> -Alex
>>>>>> 
>>>>>>> On Sep 28, 2015, at 5:02 PM, Chris Bieneman
<beanz at apple.com> wrote:
>>>>>>> 
>>>>>>> I think I understand what you think the issue is. I
would have assumed that the generated makefiles could handle two targets
building simultaneously, but you may be right.
>>>>>>> 
>>>>>>> I don’t think we should solve this with separate
directories though because that is a waste. A better solution would be making a
custom command & target that builds the native libSupport, then make all
tablegen targets depend on that one.
>>>>>>> 
>>>>>>> That way we only build the native libSupport once.
>>>>>>> 
>>>>>>> -Chris
>>>>>>> 
>>>>>>>> On Sep 28, 2015, at 1:48 PM, Alex Wang
<aw1621107 at gmail.com> wrote:
>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Sep 28, 2015, at 3:56 PM, Chris Bieneman
<beanz at apple.com> wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Sep 28, 2015, at 12:07 PM, Alex Wang
via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>>>>> 
>>>>>>>>>> Hello backend devs,
>>>>>>>>>> 
>>>>>>>>>> Been working on a parallelization bug
in the cmake configs, where parallel makefile builds of llvm with clang and
native tablegen would usually break during linking in either either the
clang_tblgen or llvm_tblgen targets. Looking at the cmake command that generates
the tablegen makefile commands (I think) (cmake/modules/TableGen.cmake:113-117):
>>>>>>>>>> 
>>>>>>>>>> add_custom_command(OUTPUT
${${project}_TABLEGEN_EXE}
>>>>>>>>>>  COMMAND ${CMAKE_COMMAND} --build ../..
--target ${target} --config Release
>>>>>>>>>>  DEPENDS CONFIGURE_LLVM_NATIVE
${target}
>>>>>>>>>>  WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>>>>>>>  COMMENT "Building native
TableGen...")
>>>>>>>>>> 
>>>>>>>>>> My guess is that the clang and llvm
tablegen builds are trying to place their output into the same folder, so
depending on when the resulting library gets linked/moved one of the threads
could be left dangling with a partially-completed library or no library.
>>>>>>>>> 
>>>>>>>>> If you build with LLVM_OPTIMIZED_TABLEGEN
the native tablegen is a fully generated build subtree. Make seems to have some
problems with thread scheduling with these builds. I haven’t diagnosed the
issue, but it isn’t that the builds are writing to the same directory. It
happens regardless of whether you’re building llvm with or without clang, and
the llvm & clang tablegen executables are named differently so they don’t
conflict.
>>>>>>>> 
>>>>>>>> Hmmm, that's odd. I never saw the build
break like this when building only llvm. Seems I just got really (un)lucky...
>>>>>>>> 
>>>>>>>> Probably could have explained this better... It
wasn't that I thought the output executables would clash; I thought that
both the llvm and clang tablegen builds were working on libLLVMSupport at the
same time in the same place, so when one finished and moved the library to its
proper place the other thread is left without anything to work with. Seems like
what you said would still apply though.
>>>>>>>> 
>>>>>>>> Another thing to note is that when building
with clang the cmake percentages for that particular library would always go
above 100%. I thought that meant that they could be writing to the same progress
file, because that never happened with llvm-only builds.
>>>>>>>> 
>>>>>>>>>> The error was either "Couldn't
stat output file: ../LLVMSupport.a No such file or directory" or something
about an archive member extending past the end of the file, and the cmake
progress messages before linking were printed twice. Once for each thread,
I'd guess? Sort of surprised that they seem to share so much code (or at
least filenames)
>>>>>>>>>> 
>>>>>>>>>> First thing I could think of to fix
this is adding another layer or two of CMakeLists, so the tablegen creation
command would be invoked in different folders. I'm not convinced that that
is the best fix, and I couldn't figure out how to move the builds into
different folders with only minor changes to the existing cmake files.
>>>>>>>>> 
>>>>>>>>> I don’t think that will work. As I said
above, they are building to different directories. You can note that in the
custom command setting WORKING_DIRECTORY to ${LLVM_NATIVE_BUILD}, which is a
nested fully-independent cmake build tree.
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> At least on my machine, ${LLVM_NATIVE_BUILD} is
the same for both llvm and clang tablegen commands.
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Thoughts?
>>>>>>>>>> 
>>>>>>>>>> -Alex
>>>>>>>>>> 
>>>>>>>>>> (CrossCompile.cmake also has a typo in
the first line: "toochain" --> "toolchain", if that is
something to worry about)
>>>>>>>>>>
_______________________________________________
>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> 
>> 
>

Alex Wang via llvm-dev

2015-Oct-08 22:15 UTC

head link

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

Alright, this version works for me.

Anything else that needs to be done?

-Alex
> On Oct 7, 2015, at 8:15 PM, Alex Wang <aw1621107 at gmail.com> wrote:
> 
> diff --git a/cmake/modules/TableGen.cmake b/cmake/modules/TableGen.cmake
> index 452a728..cb06450 100644
> --- a/cmake/modules/TableGen.cmake
> +++ b/cmake/modules/TableGen.cmake
> @@ -70,6 +70,15 @@ function(add_public_tablegen_target target)
>   set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} ${target} PARENT_SCOPE)
> endfunction()
> 
> +if(LLVM_USE_HOST_TOOLS)
> +  add_custom_command(OUTPUT LIB_LLVMSUPPORT
> +      COMMAND ${CMAKE_COMMAND} --build . --target LLVMSupport --config
Release
> +      DEPENDS CONFIGURE_LLVM_NATIVE
> +      WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
> +      COMMENT "Building libLLVMSupport for native TableGen...")
> +  add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS LIB_LLVMSUPPORT)
> +endif(LLVM_USE_HOST_TOOLS)
> +
> macro(add_tablegen target project)
>   set(${target}_OLD_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
>   set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
> @@ -109,7 +118,7 @@ macro(add_tablegen target project)
> 
>       add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>         COMMAND ${CMAKE_COMMAND} --build . --target ${target} --config
Release
> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>         WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>         COMMENT "Building native TableGen...")
>       add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
> 
> The first version definitely worked, but I can't confirm that this
version works because I'm getting compiler segfaults in Apple's clang
when trying to compile... Can't take a close look right now, but I
definitely will sometime tonight.
> 
> -Alex
> 
>> On Oct 7, 2015, at 7:18 PM, Chris Bieneman <beanz at apple.com>
wrote:
>> 
>> It should probably be inside an `if(LLVM_USE_HOST_TOOLS)` block.
>> 
>> That way the extra target and command won’t get added unless needed and
CMake won’t spew dev warnings. Adding a target with a nonexistent dependency
makes CMake dump a bunch of developer warnings.
>> 
>> Other than that this looks good. Does it resolve the issue for you?
>> 
>> -Chris
>> 
>>> On Oct 7, 2015, at 4:09 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>> 
>>> diff --git a/cmake/modules/TableGen.cmake
b/cmake/modules/TableGen.cmake
>>> index 452a728..b24197b 100644
>>> --- a/cmake/modules/TableGen.cmake
>>> +++ b/cmake/modules/TableGen.cmake
>>> @@ -70,6 +70,13 @@ function(add_public_tablegen_target target)
>>> set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} ${target}
PARENT_SCOPE)
>>> endfunction()
>>> 
>>> +add_custom_command(OUTPUT LIB_LLVMSUPPORT
>>> +    COMMAND ${CMAKE_COMMAND} --build . --target LLVMSupport
--config Release
>>> +    DEPENDS CONFIGURE_LLVM_NATIVE
>>> +    WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>> +    COMMENT "Building libLLVMSupport for native
TableGen...")
>>> +add_custom_target(NATIVE_LIB_LLVMSUPPORT DEPENDS LIB_LLVMSUPPORT)
>>> +
>>> macro(add_tablegen target project)
>>> set(${target}_OLD_LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS})
>>> set(LLVM_LINK_COMPONENTS ${LLVM_LINK_COMPONENTS} TableGen)
>>> @@ -109,7 +116,7 @@ macro(add_tablegen target project)
>>> 
>>>     add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>>>       COMMAND ${CMAKE_COMMAND} --build . --target ${target}
--config Release
>>> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>>> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>>>       WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>       COMMENT "Building native TableGen...")
>>>     add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
>>> 
>>> Something like this?
>>> 
>>>> On Oct 6, 2015, at 7:58 PM, Chris Bieneman <beanz at
apple.com> wrote:
>>>> 
>>>> 
>>>>> On Oct 5, 2015, at 9:37 AM, Alex Wang via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>>>>> 
>>>>> Probably should have checked earlier... Do patches go
directly to llvm-commits for review instead of getting reviewed here first?
>>>>> 
>>>>> ccing llvm-commits too in case
>>>>>> On Oct 4, 2015, at 8:20 PM, Alex Wang <aw1621107 at
gmail.com> wrote:
>>>>>> 
>>>>>> Alright, got something thrown together. Here it is
inline (also attached if that's more convenient):
>>>>>> 
>>>>>> diff --git a/cmake/modules/TableGen.cmake
b/cmake/modules/TableGen.cmake
>>>>>> index 452a728..be6729d 100644
>>>>>> --- a/cmake/modules/TableGen.cmake
>>>>>> +++ b/cmake/modules/TableGen.cmake
>>>>>> @@ -107,9 +107,18 @@ macro(add_tablegen target project)
>>>>>>  endif()
>>>>>>  set(${project}_TABLEGEN_EXE ${${project}_TABLEGEN_EXE}
PARENT_SCOPE)
>>>>>> 
>>>>>> +      if(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>>>> 
>>>> Instead of doing this you could move creating this out of the
macro. Then it will be executed when the file is included which should only be
once.
>>>> 
>>>>>> +        add_custom_command(OUTPUT LIB_LLVMSUPPORT
>>>>>> +          COMMAND ${CMAKE_COMMAND} --build . --target
LLVMSupport --config Release
>>>>>> +          DEPENDS CONFIGURE_LLVM_NATIVE
>>>>>> +          WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>>> +          COMMENT "Building libLLVMSupport for
native TableGen...")
>>>>>> +        add_custom_target(NATIVE_LIB_LLVMSUPPORT
DEPENDS LIB_LLVMSUPPORT)
>>>>>> +      endif(NOT TARGET NATIVE_LIB_LLVMSUPPORT)
>>>>>> +
>>>>>>  add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}
>>>>>>    COMMAND ${CMAKE_COMMAND} --build . --target
${target} --config Release
>>>>>> -        DEPENDS CONFIGURE_LLVM_NATIVE ${target}
>>>>>> +        DEPENDS ${target} NATIVE_LIB_LLVMSUPPORT
>>>>>>    WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}
>>>>>>    COMMENT "Building native TableGen...")
>>>>>>  add_custom_target(${project}-tablegen-host DEPENDS
${${project}_TABLEGEN_EXE})
>>>>>> 
>>>>>> Basically does exactly what Chris suggested: Make a
custom command + target just for the native libLLVMSupport, and have the
existing stuff depend on that.
>>>>>> 
>>>>>> I don't really like having to check to see if
NATIVE_LIB_LLVMSUPPORT is already defined as a target, but I'm not sure what
else to do. It looks like this particular macro is called multiple times, and I
don't think defining this elsewhere wouldn't be very good because
it's only used here. It shouldn't have a significant effect, though,
because the cmake prep is so short compared to the actual build.
>>>>>> 
>>>>>> -Alex
>>>>>> 
>>>>>> <TableGen.cmake.patch>
>>>>>>> On Sep 28, 2015, at 5:19 PM, Alex Wang
<aw1621107 at gmail.com> wrote:
>>>>>>> 
>>>>>>> That sounds much more reasonable than more
directories. I'll see if I can't hack something together...
>>>>>>> 
>>>>>>> I still think you have a point; I would have
expected cmake to catch that dependency. Strange thing is that just that library
had an independent cmake build going, with different percents showing and not
being verbose when the main config had it on.
>>>>>>> 
>>>>>>> -Alex
>>>>>>> 
>>>>>>>> On Sep 28, 2015, at 5:02 PM, Chris Bieneman
<beanz at apple.com> wrote:
>>>>>>>> 
>>>>>>>> I think I understand what you think the issue
is. I would have assumed that the generated makefiles could handle two targets
building simultaneously, but you may be right.
>>>>>>>> 
>>>>>>>> I don’t think we should solve this with
separate directories though because that is a waste. A better solution would be
making a custom command & target that builds the native libSupport, then
make all tablegen targets depend on that one.
>>>>>>>> 
>>>>>>>> That way we only build the native libSupport
once.
>>>>>>>> 
>>>>>>>> -Chris
>>>>>>>> 
>>>>>>>>> On Sep 28, 2015, at 1:48 PM, Alex Wang
<aw1621107 at gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Sep 28, 2015, at 3:56 PM, Chris
Bieneman <beanz at apple.com> wrote:
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Sep 28, 2015, at 12:07 PM, Alex
Wang via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hello backend devs,
>>>>>>>>>>> 
>>>>>>>>>>> Been working on a parallelization
bug in the cmake configs, where parallel makefile builds of llvm with clang and
native tablegen would usually break during linking in either either the
clang_tblgen or llvm_tblgen targets. Looking at the cmake command that generates
the tablegen makefile commands (I think) (cmake/modules/TableGen.cmake:113-117):
>>>>>>>>>>> 
>>>>>>>>>>> add_custom_command(OUTPUT
${${project}_TABLEGEN_EXE}
>>>>>>>>>>> COMMAND ${CMAKE_COMMAND} --build
../.. --target ${target} --config Release
>>>>>>>>>>> DEPENDS CONFIGURE_LLVM_NATIVE
${target}
>>>>>>>>>>> WORKING_DIRECTORY
${LLVM_NATIVE_BUILD}
>>>>>>>>>>> COMMENT "Building native
TableGen...")
>>>>>>>>>>> 
>>>>>>>>>>> My guess is that the clang and llvm
tablegen builds are trying to place their output into the same folder, so
depending on when the resulting library gets linked/moved one of the threads
could be left dangling with a partially-completed library or no library.
>>>>>>>>>> 
>>>>>>>>>> If you build with
LLVM_OPTIMIZED_TABLEGEN the native tablegen is a fully generated build subtree.
Make seems to have some problems with thread scheduling with these builds. I
haven’t diagnosed the issue, but it isn’t that the builds are writing to the
same directory. It happens regardless of whether you’re building llvm with or
without clang, and the llvm & clang tablegen executables are named
differently so they don’t conflict.
>>>>>>>>> 
>>>>>>>>> Hmmm, that's odd. I never saw the build
break like this when building only llvm. Seems I just got really (un)lucky...
>>>>>>>>> 
>>>>>>>>> Probably could have explained this
better... It wasn't that I thought the output executables would clash; I
thought that both the llvm and clang tablegen builds were working on
libLLVMSupport at the same time in the same place, so when one finished and
moved the library to its proper place the other thread is left without anything
to work with. Seems like what you said would still apply though.
>>>>>>>>> 
>>>>>>>>> Another thing to note is that when building
with clang the cmake percentages for that particular library would always go
above 100%. I thought that meant that they could be writing to the same progress
file, because that never happened with llvm-only builds.
>>>>>>>>> 
>>>>>>>>>>> The error was either
"Couldn't stat output file: ../LLVMSupport.a No such file or
directory" or something about an archive member extending past the end of
the file, and the cmake progress messages before linking were printed twice.
Once for each thread, I'd guess? Sort of surprised that they seem to share
so much code (or at least filenames)
>>>>>>>>>>> 
>>>>>>>>>>> First thing I could think of to fix
this is adding another layer or two of CMakeLists, so the tablegen creation
command would be invoked in different folders. I'm not convinced that that
is the best fix, and I couldn't figure out how to move the builds into
different folders with only minor changes to the existing cmake files.
>>>>>>>>>> 
>>>>>>>>>> I don’t think that will work. As I said
above, they are building to different directories. You can note that in the
custom command setting WORKING_DIRECTORY to ${LLVM_NATIVE_BUILD}, which is a
nested fully-independent cmake build tree.
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> At least on my machine,
${LLVM_NATIVE_BUILD} is the same for both llvm and clang tablegen commands.
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Thoughts?
>>>>>>>>>>> 
>>>>>>>>>>> -Alex
>>>>>>>>>>> 
>>>>>>>>>>> (CrossCompile.cmake also has a typo
in the first line: "toochain" --> "toolchain", if that is
something to worry about)
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>>>>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> 
>>> 
>> 
>

Possibly Parallel Threads

Search for more maybe matching threads

llvm dev - Oct 2015 - Cmake-gen'd parallel make breaks on native tablegen

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

[llvm-dev] Cmake-gen'd parallel make breaks on native tablegen

Possibly Parallel Threads