> On Jul 29, 2015, at 7:03 PM, Tom Stellard <tom at stellard.net> wrote: > > On Wed, Jul 29, 2015 at 01:36:43PM -0700, Chris Bieneman wrote: >> Hi LLVMDev, >> >> I wanted to take some time to write up and roll out a proposed road map for CMake over the next few months. Apologies in advance for the substantial 0/7 Wall of Text I've summoned here (it only cost me 3 swamp mana). >> >> The main thing I want to talk about is PR21562. For Apple PR21562 is the biggest reason we can't fully abandon autoconf. I've spent some time over the last month hacking on compiler-rt's build system and I've come up with a basic outline of the approach I'd like to take. >> >> (1) Reconcile out-of-tree functionality >> >> Apple has a little bit of out-of-tree code that adds functionality to compiler-rt's CMake build system. Specifically the ability to build for iOS. Before I can do any substantial refactoring of the open source build system I need to bring it to feature parity with our internal system. I have three remaining patches for that. >> >> D11083 [CMake] Add experimental support for building compiler-rt for iOS >> D11082 [CMake] Adding some utility functions for Darwin builds into a new CompilerRTDarwinUtils.cmake module >> D11073 Architectures for darwin need to be conditionalized based on the operating system. >> >> One thing worth noting here. I need to land some variant of this functionality, but the way this is done is hacky and should go away. Once these are landed I can begin the more substantial reworking. >> >> (2) Build the new Darwin build behavior >> >> One of the big problems with building compiler-rt for Darwin is that the CMake has been hacked to build multiple platforms at once (yes I know my patches above only make this worse). The first thing I want to do is add a new variable (something like COMPILER_RT_EXPERIMENTAL_DARWIN_BUILD) to toggle into a new mode for building on Darwin. >> >> The new Darwin build mode will build a single triple at a time in much the same way the Android and Linux builds work. In fact, my plan is to have the new Darwin functionality almost completely follow the Linux and Android code paths. >> >> There is a healthy bit of hand waving here because this task is larger than I'm making it sound. Compiler-rt is comprised of two very different sets of components, and they have some significantly different requirements. A big part of this work is going to be the root of PR21562, replacing the specific functionality in make/platform/clang_darwin.mk. That functionality drives building the runtime pieces of compiler-rt. There will need to be similar, but slightly different work done for the sanitizers. >> >> (3) Make compiler-rt an external project when built in-tree with Clang >> >> Once compiler-rt builds for one and only one target at a time, we will need to hook that into the LLVM build system so that you can do in-tree builds of compiler-rt for all the targets you can target with your new clang. I will gate this functionality on a flag just like the work above so it won't break existing users. >> >> This re-working will also provide a more robust solution for PR14109. PR14109 can't be properly fixed without making compiler-rt an external project because CMake doesn't really support changing the compiler after you've already configured a directory. >> >> Some of this functionality already exists in clang/runtime, but we'll want to transition to that being the only supported way to build compiler-rt, and (for Darwin) we'll need to add some support for lipo-ing thin binaries together. >> >> (4) Celebrate! >> >> That's really the hard bits. Once we get the new behavior working through the stack we'll need to test it like nobody's business and then we can talk about making it default and cleaning up all the code that supported the old way of doing things. >> >> So... what's next? >> >> With PR14109 and PR21562 done Apple will be able to migrate off autoconf, and there will only be a few currently identified issues preventing CMake from replacing autoconf. Tackling the remaining issues should be fairly straight forward as none of them are too gnarly. >> >> Moving beyond this all I have some ideas for improving our CMake scripts to make developers more productive, and I have some ideas for improvements that we could get by working with the CMake community. For example, I want our CMake build system to have convenience targets for things that we currently drive with out-of-tree tooling. This includes clang bootstrap builds, and generating PGO data and linker order files. I also would like to work with the CMake community to get a way to wire up external projects so that they could all be mapped into a single Ninja build file for better parallelism and faster incremental builds. >> >> Questions, comments, concerns, condolences? >> > > Are we close enough to fixing all the CMake bugs that we can mark autoconf > as deprecated now and plan to remove it after the 3.8 release? > > -TomThat is premature. The compiler-rt build is not even close to parity with the makefiles (at least for Apple platforms). Chris is working on a road map to fix that, but we’re still at the beginning of that road, not the end.> >> Thanks, >> -Chris >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150729/3194d78c/attachment.html>
On Wed, Jul 29, 2015 at 07:17:13PM -0700, Bob Wilson wrote:> > > On Jul 29, 2015, at 7:03 PM, Tom Stellard <tom at stellard.net> wrote: > > > > On Wed, Jul 29, 2015 at 01:36:43PM -0700, Chris Bieneman wrote: > >> Hi LLVMDev, > >> > >> I wanted to take some time to write up and roll out a proposed road map for CMake over the next few months. Apologies in advance for the substantial 0/7 Wall of Text I've summoned here (it only cost me 3 swamp mana). > >> > >> The main thing I want to talk about is PR21562. For Apple PR21562 is the biggest reason we can't fully abandon autoconf. I've spent some time over the last month hacking on compiler-rt's build system and I've come up with a basic outline of the approach I'd like to take. > >> > >> (1) Reconcile out-of-tree functionality > >> > >> Apple has a little bit of out-of-tree code that adds functionality to compiler-rt's CMake build system. Specifically the ability to build for iOS. Before I can do any substantial refactoring of the open source build system I need to bring it to feature parity with our internal system. I have three remaining patches for that. > >> > >> D11083 [CMake] Add experimental support for building compiler-rt for iOS > >> D11082 [CMake] Adding some utility functions for Darwin builds into a new CompilerRTDarwinUtils.cmake module > >> D11073 Architectures for darwin need to be conditionalized based on the operating system. > >> > >> One thing worth noting here. I need to land some variant of this functionality, but the way this is done is hacky and should go away. Once these are landed I can begin the more substantial reworking. > >> > >> (2) Build the new Darwin build behavior > >> > >> One of the big problems with building compiler-rt for Darwin is that the CMake has been hacked to build multiple platforms at once (yes I know my patches above only make this worse). The first thing I want to do is add a new variable (something like COMPILER_RT_EXPERIMENTAL_DARWIN_BUILD) to toggle into a new mode for building on Darwin. > >> > >> The new Darwin build mode will build a single triple at a time in much the same way the Android and Linux builds work. In fact, my plan is to have the new Darwin functionality almost completely follow the Linux and Android code paths. > >> > >> There is a healthy bit of hand waving here because this task is larger than I'm making it sound. Compiler-rt is comprised of two very different sets of components, and they have some significantly different requirements. A big part of this work is going to be the root of PR21562, replacing the specific functionality in make/platform/clang_darwin.mk. That functionality drives building the runtime pieces of compiler-rt. There will need to be similar, but slightly different work done for the sanitizers. > >> > >> (3) Make compiler-rt an external project when built in-tree with Clang > >> > >> Once compiler-rt builds for one and only one target at a time, we will need to hook that into the LLVM build system so that you can do in-tree builds of compiler-rt for all the targets you can target with your new clang. I will gate this functionality on a flag just like the work above so it won't break existing users. > >> > >> This re-working will also provide a more robust solution for PR14109. PR14109 can't be properly fixed without making compiler-rt an external project because CMake doesn't really support changing the compiler after you've already configured a directory. > >> > >> Some of this functionality already exists in clang/runtime, but we'll want to transition to that being the only supported way to build compiler-rt, and (for Darwin) we'll need to add some support for lipo-ing thin binaries together. > >> > >> (4) Celebrate! > >> > >> That's really the hard bits. Once we get the new behavior working through the stack we'll need to test it like nobody's business and then we can talk about making it default and cleaning up all the code that supported the old way of doing things. > >> > >> So... what's next? > >> > >> With PR14109 and PR21562 done Apple will be able to migrate off autoconf, and there will only be a few currently identified issues preventing CMake from replacing autoconf. Tackling the remaining issues should be fairly straight forward as none of them are too gnarly. > >> > >> Moving beyond this all I have some ideas for improving our CMake scripts to make developers more productive, and I have some ideas for improvements that we could get by working with the CMake community. For example, I want our CMake build system to have convenience targets for things that we currently drive with out-of-tree tooling. This includes clang bootstrap builds, and generating PGO data and linker order files. I also would like to work with the CMake community to get a way to wire up external projects so that they could all be mapped into a single Ninja build file for better parallelism and faster incremental builds. > >> > >> Questions, comments, concerns, condolences? > >> > > > > Are we close enough to fixing all the CMake bugs that we can mark autoconf > > as deprecated now and plan to remove it after the 3.8 release? > > > > -Tom > > That is premature. The compiler-rt build is not even close to parity with the makefiles (at least for Apple platforms). Chris is working on a road map to fix that, but we’re still at the beginning of that road, not the end. >Is it possible to move to CMake earlier on projects besides comiler-rt, or does compiler-rt require building llvm with autoconf? -Tom> > > >> Thanks, > >> -Chris > >> _______________________________________________ > >> LLVM Developers mailing list > >> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> > >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
> On Jul 30, 2015, at 7:35 AM, Tom Stellard <tom at stellard.net> wrote: > > On Wed, Jul 29, 2015 at 07:17:13PM -0700, Bob Wilson wrote: >> >>> On Jul 29, 2015, at 7:03 PM, Tom Stellard <tom at stellard.net> wrote: >>> >>> On Wed, Jul 29, 2015 at 01:36:43PM -0700, Chris Bieneman wrote: >>>> Hi LLVMDev, >>>> >>>> I wanted to take some time to write up and roll out a proposed road map for CMake over the next few months. Apologies in advance for the substantial 0/7 Wall of Text I've summoned here (it only cost me 3 swamp mana). >>>> >>>> The main thing I want to talk about is PR21562. For Apple PR21562 is the biggest reason we can't fully abandon autoconf. I've spent some time over the last month hacking on compiler-rt's build system and I've come up with a basic outline of the approach I'd like to take. >>>> >>>> (1) Reconcile out-of-tree functionality >>>> >>>> Apple has a little bit of out-of-tree code that adds functionality to compiler-rt's CMake build system. Specifically the ability to build for iOS. Before I can do any substantial refactoring of the open source build system I need to bring it to feature parity with our internal system. I have three remaining patches for that. >>>> >>>> D11083 [CMake] Add experimental support for building compiler-rt for iOS >>>> D11082 [CMake] Adding some utility functions for Darwin builds into a new CompilerRTDarwinUtils.cmake module >>>> D11073 Architectures for darwin need to be conditionalized based on the operating system. >>>> >>>> One thing worth noting here. I need to land some variant of this functionality, but the way this is done is hacky and should go away. Once these are landed I can begin the more substantial reworking. >>>> >>>> (2) Build the new Darwin build behavior >>>> >>>> One of the big problems with building compiler-rt for Darwin is that the CMake has been hacked to build multiple platforms at once (yes I know my patches above only make this worse). The first thing I want to do is add a new variable (something like COMPILER_RT_EXPERIMENTAL_DARWIN_BUILD) to toggle into a new mode for building on Darwin. >>>> >>>> The new Darwin build mode will build a single triple at a time in much the same way the Android and Linux builds work. In fact, my plan is to have the new Darwin functionality almost completely follow the Linux and Android code paths. >>>> >>>> There is a healthy bit of hand waving here because this task is larger than I'm making it sound. Compiler-rt is comprised of two very different sets of components, and they have some significantly different requirements. A big part of this work is going to be the root of PR21562, replacing the specific functionality in make/platform/clang_darwin.mk. That functionality drives building the runtime pieces of compiler-rt. There will need to be similar, but slightly different work done for the sanitizers. >>>> >>>> (3) Make compiler-rt an external project when built in-tree with Clang >>>> >>>> Once compiler-rt builds for one and only one target at a time, we will need to hook that into the LLVM build system so that you can do in-tree builds of compiler-rt for all the targets you can target with your new clang. I will gate this functionality on a flag just like the work above so it won't break existing users. >>>> >>>> This re-working will also provide a more robust solution for PR14109. PR14109 can't be properly fixed without making compiler-rt an external project because CMake doesn't really support changing the compiler after you've already configured a directory. >>>> >>>> Some of this functionality already exists in clang/runtime, but we'll want to transition to that being the only supported way to build compiler-rt, and (for Darwin) we'll need to add some support for lipo-ing thin binaries together. >>>> >>>> (4) Celebrate! >>>> >>>> That's really the hard bits. Once we get the new behavior working through the stack we'll need to test it like nobody's business and then we can talk about making it default and cleaning up all the code that supported the old way of doing things. >>>> >>>> So... what's next? >>>> >>>> With PR14109 and PR21562 done Apple will be able to migrate off autoconf, and there will only be a few currently identified issues preventing CMake from replacing autoconf. Tackling the remaining issues should be fairly straight forward as none of them are too gnarly. >>>> >>>> Moving beyond this all I have some ideas for improving our CMake scripts to make developers more productive, and I have some ideas for improvements that we could get by working with the CMake community. For example, I want our CMake build system to have convenience targets for things that we currently drive with out-of-tree tooling. This includes clang bootstrap builds, and generating PGO data and linker order files. I also would like to work with the CMake community to get a way to wire up external projects so that they could all be mapped into a single Ninja build file for better parallelism and faster incremental builds. >>>> >>>> Questions, comments, concerns, condolences? >>>> >>> >>> Are we close enough to fixing all the CMake bugs that we can mark autoconf >>> as deprecated now and plan to remove it after the 3.8 release?Tom, I think it is a bit too soon to mark it as deprecated, but I hope to be in a position to mark it as deprecated sometime this fall. So hopefully, yes it will be deprecated before the 3.8 branch and hopefully we can remove it after the 3.8 release. That said, I think there is still a healthy bit of work before we get to a point where I am comfortable marking it as deprecated. My hope at this point is that the community will at least mostly agree with the plan I laid out at the start of this thread, and that we can make some rapid progress over the next few months.>>> >>> -Tom >> >> That is premature. The compiler-rt build is not even close to parity with the makefiles (at least for Apple platforms). Chris is working on a road map to fix that, but we’re still at the beginning of that road, not the end. >> > > Is it possible to move to CMake earlier on projects besides comiler-rt, > or does compiler-rt require building llvm with autoconf?I’m sure it is possible to make CMake call into compiler-rt’s autoconf build, but compiler-rt’s autoconf build system is not feature compatible against its CMake build system, and I think making this work reliably would be more work than it is worth. -Chris> > -Tom > >>> >>>> Thanks, >>>> -Chris >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> <mailto:LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> <http://llvm.cs.uiuc.edu/ <http://llvm.cs.uiuc.edu/>> >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> <mailto:LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>> http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/> <http://llvm.cs.uiuc.edu/ <http://llvm.cs.uiuc.edu/>> >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150730/9ceeac81/attachment.html>