Fabio Pagani via llvm-dev
2016-Mar-02 21:28 UTC
[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS
Hey Chris, Sedat was asking for a way to "to speedup my build" and those blog posts were really helpful to me. Anyway LLVM_DISTRIBUTION_COMPONENTS sounds very cool, hope you will push your code soon! On Tue, Mar 1, 2016 at 11:32 PM, Chris Bieneman <cbieneman at apple.com> wrote:> Fabio, the work I was mentioning here is an extension beyond those blog > posts. > > Some details: > > * The “almost 40%” number I referred to is a multi-stage clang build. That > means we build a host-capable compiler, then build the actual compiler we > want to ship. > * I’m at Apple, so points 1 and 2 are already covered (we only use clang, > and ld64 is a fast linker). > * Our system compiler is PGO+LTO’d, but our stage1 isn’t. Stage1 isn’t > because the performance improvement of PGO+LTO is less than the time it > takes to build, and stage1 is basically a throwaway. > * We are using Ninja and CMake, but this configuration isn’t really > significantly faster than autoconf/make, and actually “ninja install” is > slower in my tests than the old autoconf “make install”. The slowdown is > almost entirely due to Ninja’s “all” target being a lot bigger. > * This performance is for clean builds, not incremental so ccache or > shared libraries would not be a valid optimization > * We do use optimized tablegen > * “Build Less” is exactly what the LLVM_DISTRIBUTION_COMPONENTS enables, > just in a friendly wrapper target. > > -Chris > > > > On Mar 1, 2016, at 1:12 PM, Fabio Pagani <pagabuc at gmail.com> wrote: > > For faster builds and rebuilds you should definitely read: > > https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/ > https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/ > > Hope this helps! > > On Tue, Mar 1, 2016 at 9:17 PM, ChrisBieneman via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> >> >> > On Mar 1, 2016, at 10:01 AM, Mehdi Amini <mehdi.amini at apple.com> wrote: >> > >> > >> >> On Mar 1, 2016, at 9:57 AM, Chris Bieneman <cbieneman at apple.com> >> wrote: >> >> >> >> There are a few notes I'd like to add to this thread. >> >> >> >> (1) we have a number of places throughout out CMake build where we use >> features from newer CMakes gated by version checks. Most of these features >> are performance or usability related. None of them are correctness. Using >> the latest CMake release will often result in faster builds, so I encourage >> it. >> >> >> >> (2) CMake's "install" target will pretty much always be slower from >> clean than the old autoconf/make "install" target. This is because in CMake >> "install" depends on "all", and our CMake builds more stuff in "all" than >> autoconf did. To help with this or CMake system has lots of convenient >> "install-${name}" targets that support component-based installation. Not >> every component has one of these rules, but if one you need is missing let >> me know. I also recently (r261681) added a new option >> (LLVM_DISTRIBUTION_COMPONENTS) that allows you to specify a list of >> components that have custom install targets. It then creates a new >> "install-distribution" target that wraps just the components you want. For >> Apple this is almost a 40% speed up over "ninja install". >> > >> > That sounds great, I want to use it! >> > It would even be more awesome with an description/example in >> docs/CMake.rst :) >> >> Once I get the last of the kinks worked out for our internal adoption I'm >> going to open source our config files that use it. >> >> I've also made a note to remind myself to document it in docs/CMake.rst. >> I need to do a pass updating that with a bunch of the cool new things we're >> doing with CMake. Thanks for the reminder. >> >> -Chris >> >> > >> > -- >> > Mehdi >> >> >> >> On Feb 25, 2016, at 11:08 AM, Sedat Dilek via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >> >> >>>> Which combination of cmake/ninja versions are you using (latest are >> >>>> v3.4.3 and v1.6.0)? >> >>> >> >>> With this combination I could reduce build-time down from approx. 3h >> >>> down to 01h20m. >> >>> >> >>> $ egrep -i 'jobs|ninja' llvm-build/CMakeCache.txt >> >>> //Program used to build from build.ninja files. >> >>> CMAKE_MAKE_PROGRAM:FILEPATH=/opt/cmake/bin/ninja >> >>> //Define the maximum number of concurrent compilation jobs. >> >>> LLVM_PARALLEL_COMPILE_JOBS:STRING=3 >> >>> //Define the maximum number of concurrent link jobs. >> >>> LLVM_PARALLEL_LINK_JOBS:STRING=1 >> >>> CMAKE_GENERATOR:INTERNAL=Ninja >> >>> >> >>> $ LC_ALL=C ls -alt >> logs/3.8.0rc3_clang-3-8-0-rc3_cmake-3-4-3_ninja-1-6-0/ >> >>> total 360 >> >>> drwxr-xr-x 2 wearefam wearefam 4096 Feb 25 19:58 . >> >>> drwxr-xr-x 6 wearefam wearefam 4096 Feb 25 19:58 .. >> >>> -rw-r--r-- 1 wearefam wearefam 130196 Feb 25 19:54 >> >>> install-log_llvm-toolchain-3.8.0rc3.txt >> >>> -rw-r--r-- 1 wearefam wearefam 205762 Feb 25 19:51 >> >>> build-log_llvm-toolchain-3.8.0rc3.txt >> >>> -rw-r--r-- 1 wearefam wearefam 14331 Feb 25 18:30 >> >>> configure-log_llvm-toolchain-3.8.0rc3.txt >> >>> >> >>> $ LC_ALL=C du -s -m llvm* /opt/llvm-toolchain-3.8.0rc3 >> >>> 315 llvm >> >>> 941 llvm-build >> >>> 609 /opt/llvm-toolchain-3.8.0rc3 >> >>> >> >>> - Sedat - >> >>> >> >>> [1] https://cmake.org/files/v3.5/cmake-3.5.0-rc3-Linux-x86_64.tar.gz >> >>> _______________________________________________ >> >>> LLVM Developers mailing list >> >>> llvm-dev at lists.llvm.org >> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160302/67aa4df3/attachment.html>
Sedat Dilek via llvm-dev
2016-Mar-03 00:22 UTC
[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS
I got some more inspirations on how to speedup my build and integrated the URLs into my scripts (attached). For example to use GOLD as linker or to use '-O3' OptLevel maybe in combination with LTO and PGO (using '-O3 -flto -fprofile-use'). Let's see when the v3.8.0 FINAL is released. - Sedat - On 3/2/16, Fabio Pagani <pagabuc at gmail.com> wrote:> Hey Chris, > > Sedat was asking for a way to "to speedup my build" and those blog posts > were really helpful to me. > Anyway LLVM_DISTRIBUTION_COMPONENTS sounds very cool, hope you will push > your code soon! > > On Tue, Mar 1, 2016 at 11:32 PM, Chris Bieneman <cbieneman at apple.com> > wrote: > >> Fabio, the work I was mentioning here is an extension beyond those blog >> posts. >> >> Some details: >> >> * The “almost 40%” number I referred to is a multi-stage clang build. >> That >> means we build a host-capable compiler, then build the actual compiler we >> want to ship. >> * I’m at Apple, so points 1 and 2 are already covered (we only use clang, >> and ld64 is a fast linker). >> * Our system compiler is PGO+LTO’d, but our stage1 isn’t. Stage1 isn’t >> because the performance improvement of PGO+LTO is less than the time it >> takes to build, and stage1 is basically a throwaway. >> * We are using Ninja and CMake, but this configuration isn’t really >> significantly faster than autoconf/make, and actually “ninja install” is >> slower in my tests than the old autoconf “make install”. The slowdown is >> almost entirely due to Ninja’s “all” target being a lot bigger. >> * This performance is for clean builds, not incremental so ccache or >> shared libraries would not be a valid optimization >> * We do use optimized tablegen >> * “Build Less” is exactly what the LLVM_DISTRIBUTION_COMPONENTS enables, >> just in a friendly wrapper target. >> >> -Chris >> >> >> >> On Mar 1, 2016, at 1:12 PM, Fabio Pagani <pagabuc at gmail.com> wrote: >> >> For faster builds and rebuilds you should definitely read: >> >> https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/ >> https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/ >> >> Hope this helps! >> >> On Tue, Mar 1, 2016 at 9:17 PM, ChrisBieneman via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> >>> >>> > On Mar 1, 2016, at 10:01 AM, Mehdi Amini <mehdi.amini at apple.com> >>> > wrote: >>> > >>> > >>> >> On Mar 1, 2016, at 9:57 AM, Chris Bieneman <cbieneman at apple.com> >>> wrote: >>> >> >>> >> There are a few notes I'd like to add to this thread. >>> >> >>> >> (1) we have a number of places throughout out CMake build where we >>> >> use >>> features from newer CMakes gated by version checks. Most of these >>> features >>> are performance or usability related. None of them are correctness. >>> Using >>> the latest CMake release will often result in faster builds, so I >>> encourage >>> it. >>> >> >>> >> (2) CMake's "install" target will pretty much always be slower from >>> clean than the old autoconf/make "install" target. This is because in >>> CMake >>> "install" depends on "all", and our CMake builds more stuff in "all" >>> than >>> autoconf did. To help with this or CMake system has lots of convenient >>> "install-${name}" targets that support component-based installation. Not >>> every component has one of these rules, but if one you need is missing >>> let >>> me know. I also recently (r261681) added a new option >>> (LLVM_DISTRIBUTION_COMPONENTS) that allows you to specify a list of >>> components that have custom install targets. It then creates a new >>> "install-distribution" target that wraps just the components you want. >>> For >>> Apple this is almost a 40% speed up over "ninja install". >>> > >>> > That sounds great, I want to use it! >>> > It would even be more awesome with an description/example in >>> docs/CMake.rst :) >>> >>> Once I get the last of the kinks worked out for our internal adoption >>> I'm >>> going to open source our config files that use it. >>> >>> I've also made a note to remind myself to document it in docs/CMake.rst. >>> I need to do a pass updating that with a bunch of the cool new things >>> we're >>> doing with CMake. Thanks for the reminder. >>> >>> -Chris >>> >>> > >>> > -- >>> > Mehdi >>> >> >>> >> On Feb 25, 2016, at 11:08 AM, Sedat Dilek via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >> >>> >>>> Which combination of cmake/ninja versions are you using (latest are >>> >>>> v3.4.3 and v1.6.0)? >>> >>> >>> >>> With this combination I could reduce build-time down from approx. 3h >>> >>> down to 01h20m. >>> >>> >>> >>> $ egrep -i 'jobs|ninja' llvm-build/CMakeCache.txt >>> >>> //Program used to build from build.ninja files. >>> >>> CMAKE_MAKE_PROGRAM:FILEPATH=/opt/cmake/bin/ninja >>> >>> //Define the maximum number of concurrent compilation jobs. >>> >>> LLVM_PARALLEL_COMPILE_JOBS:STRING=3 >>> >>> //Define the maximum number of concurrent link jobs. >>> >>> LLVM_PARALLEL_LINK_JOBS:STRING=1 >>> >>> CMAKE_GENERATOR:INTERNAL=Ninja >>> >>> >>> >>> $ LC_ALL=C ls -alt >>> logs/3.8.0rc3_clang-3-8-0-rc3_cmake-3-4-3_ninja-1-6-0/ >>> >>> total 360 >>> >>> drwxr-xr-x 2 wearefam wearefam 4096 Feb 25 19:58 . >>> >>> drwxr-xr-x 6 wearefam wearefam 4096 Feb 25 19:58 .. >>> >>> -rw-r--r-- 1 wearefam wearefam 130196 Feb 25 19:54 >>> >>> install-log_llvm-toolchain-3.8.0rc3.txt >>> >>> -rw-r--r-- 1 wearefam wearefam 205762 Feb 25 19:51 >>> >>> build-log_llvm-toolchain-3.8.0rc3.txt >>> >>> -rw-r--r-- 1 wearefam wearefam 14331 Feb 25 18:30 >>> >>> configure-log_llvm-toolchain-3.8.0rc3.txt >>> >>> >>> >>> $ LC_ALL=C du -s -m llvm* /opt/llvm-toolchain-3.8.0rc3 >>> >>> 315 llvm >>> >>> 941 llvm-build >>> >>> 609 /opt/llvm-toolchain-3.8.0rc3 >>> >>> >>> >>> - Sedat - >>> >>> >>> >>> [1] https://cmake.org/files/v3.5/cmake-3.5.0-rc3-Linux-x86_64.tar.gz >>> >>> _______________________________________________ >>> >>> LLVM Developers mailing list >>> >>> llvm-dev at lists.llvm.org >>> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> > >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> >> >> >-------------- next part -------------- A non-text attachment was scrubbed... Name: build_llvm-toolchain_clang-cmake-ninja.sh Type: application/x-sh Size: 3728 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160303/fa3f74c0/attachment.sh> -------------- next part -------------- A non-text attachment was scrubbed... Name: install_llvm-toolchain_clang-cmake-ninja.sh Type: application/x-sh Size: 3223 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160303/fa3f74c0/attachment-0001.sh>
Mehdi Amini via llvm-dev
2016-Mar-03 00:42 UTC
[llvm-dev] Building with LLVM_PARALLEL_XXX_JOBS
> On Mar 2, 2016, at 4:22 PM, Sedat Dilek <sedat.dilek at gmail.com> wrote: > > I got some more inspirations on how to speedup my build and integrated > the URLs into my scripts (attached). > > For example to use GOLD as linker or to use '-O3' OptLevel maybe in > combination with LTO and PGO (using '-O3 -flto -fprofile-use').LTO *will* slow down dramatically the build. -- Mehdi> > Let's see when the v3.8.0 FINAL is released. > > - Sedat - > > On 3/2/16, Fabio Pagani <pagabuc at gmail.com> wrote: >> Hey Chris, >> >> Sedat was asking for a way to "to speedup my build" and those blog posts >> were really helpful to me. >> Anyway LLVM_DISTRIBUTION_COMPONENTS sounds very cool, hope you will push >> your code soon! >> >> On Tue, Mar 1, 2016 at 11:32 PM, Chris Bieneman <cbieneman at apple.com> >> wrote: >> >>> Fabio, the work I was mentioning here is an extension beyond those blog >>> posts. >>> >>> Some details: >>> >>> * The “almost 40%” number I referred to is a multi-stage clang build. >>> That >>> means we build a host-capable compiler, then build the actual compiler we >>> want to ship. >>> * I’m at Apple, so points 1 and 2 are already covered (we only use clang, >>> and ld64 is a fast linker). >>> * Our system compiler is PGO+LTO’d, but our stage1 isn’t. Stage1 isn’t >>> because the performance improvement of PGO+LTO is less than the time it >>> takes to build, and stage1 is basically a throwaway. >>> * We are using Ninja and CMake, but this configuration isn’t really >>> significantly faster than autoconf/make, and actually “ninja install” is >>> slower in my tests than the old autoconf “make install”. The slowdown is >>> almost entirely due to Ninja’s “all” target being a lot bigger. >>> * This performance is for clean builds, not incremental so ccache or >>> shared libraries would not be a valid optimization >>> * We do use optimized tablegen >>> * “Build Less” is exactly what the LLVM_DISTRIBUTION_COMPONENTS enables, >>> just in a friendly wrapper target. >>> >>> -Chris >>> >>> >>> >>> On Mar 1, 2016, at 1:12 PM, Fabio Pagani <pagabuc at gmail.com> wrote: >>> >>> For faster builds and rebuilds you should definitely read: >>> >>> https://blogs.s-osg.org/an-introduction-to-accelerating-your-build-with-clang/ >>> https://blogs.s-osg.org/a-conclusion-to-accelerating-your-build-with-clang/ >>> >>> Hope this helps! >>> >>> On Tue, Mar 1, 2016 at 9:17 PM, ChrisBieneman via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> >>>> >>>>> On Mar 1, 2016, at 10:01 AM, Mehdi Amini <mehdi.amini at apple.com> >>>>> wrote: >>>>> >>>>> >>>>>> On Mar 1, 2016, at 9:57 AM, Chris Bieneman <cbieneman at apple.com> >>>> wrote: >>>>>> >>>>>> There are a few notes I'd like to add to this thread. >>>>>> >>>>>> (1) we have a number of places throughout out CMake build where we >>>>>> use >>>> features from newer CMakes gated by version checks. Most of these >>>> features >>>> are performance or usability related. None of them are correctness. >>>> Using >>>> the latest CMake release will often result in faster builds, so I >>>> encourage >>>> it. >>>>>> >>>>>> (2) CMake's "install" target will pretty much always be slower from >>>> clean than the old autoconf/make "install" target. This is because in >>>> CMake >>>> "install" depends on "all", and our CMake builds more stuff in "all" >>>> than >>>> autoconf did. To help with this or CMake system has lots of convenient >>>> "install-${name}" targets that support component-based installation. Not >>>> every component has one of these rules, but if one you need is missing >>>> let >>>> me know. I also recently (r261681) added a new option >>>> (LLVM_DISTRIBUTION_COMPONENTS) that allows you to specify a list of >>>> components that have custom install targets. It then creates a new >>>> "install-distribution" target that wraps just the components you want. >>>> For >>>> Apple this is almost a 40% speed up over "ninja install". >>>>> >>>>> That sounds great, I want to use it! >>>>> It would even be more awesome with an description/example in >>>> docs/CMake.rst :) >>>> >>>> Once I get the last of the kinks worked out for our internal adoption >>>> I'm >>>> going to open source our config files that use it. >>>> >>>> I've also made a note to remind myself to document it in docs/CMake.rst. >>>> I need to do a pass updating that with a bunch of the cool new things >>>> we're >>>> doing with CMake. Thanks for the reminder. >>>> >>>> -Chris >>>> >>>>> >>>>> -- >>>>> Mehdi >>>>>> >>>>>> On Feb 25, 2016, at 11:08 AM, Sedat Dilek via llvm-dev < >>>> llvm-dev at lists.llvm.org> wrote: >>>>>> >>>>>>>> Which combination of cmake/ninja versions are you using (latest are >>>>>>>> v3.4.3 and v1.6.0)? >>>>>>> >>>>>>> With this combination I could reduce build-time down from approx. 3h >>>>>>> down to 01h20m. >>>>>>> >>>>>>> $ egrep -i 'jobs|ninja' llvm-build/CMakeCache.txt >>>>>>> //Program used to build from build.ninja files. >>>>>>> CMAKE_MAKE_PROGRAM:FILEPATH=/opt/cmake/bin/ninja >>>>>>> //Define the maximum number of concurrent compilation jobs. >>>>>>> LLVM_PARALLEL_COMPILE_JOBS:STRING=3 >>>>>>> //Define the maximum number of concurrent link jobs. >>>>>>> LLVM_PARALLEL_LINK_JOBS:STRING=1 >>>>>>> CMAKE_GENERATOR:INTERNAL=Ninja >>>>>>> >>>>>>> $ LC_ALL=C ls -alt >>>> logs/3.8.0rc3_clang-3-8-0-rc3_cmake-3-4-3_ninja-1-6-0/ >>>>>>> total 360 >>>>>>> drwxr-xr-x 2 wearefam wearefam 4096 Feb 25 19:58 . >>>>>>> drwxr-xr-x 6 wearefam wearefam 4096 Feb 25 19:58 .. >>>>>>> -rw-r--r-- 1 wearefam wearefam 130196 Feb 25 19:54 >>>>>>> install-log_llvm-toolchain-3.8.0rc3.txt >>>>>>> -rw-r--r-- 1 wearefam wearefam 205762 Feb 25 19:51 >>>>>>> build-log_llvm-toolchain-3.8.0rc3.txt >>>>>>> -rw-r--r-- 1 wearefam wearefam 14331 Feb 25 18:30 >>>>>>> configure-log_llvm-toolchain-3.8.0rc3.txt >>>>>>> >>>>>>> $ LC_ALL=C du -s -m llvm* /opt/llvm-toolchain-3.8.0rc3 >>>>>>> 315 llvm >>>>>>> 941 llvm-build >>>>>>> 609 /opt/llvm-toolchain-3.8.0rc3 >>>>>>> >>>>>>> - Sedat - >>>>>>> >>>>>>> [1] https://cmake.org/files/v3.5/cmake-3.5.0-rc3-Linux-x86_64.tar.gz >>>>>>> _______________________________________________ >>>>>>> LLVM Developers mailing list >>>>>>> llvm-dev at lists.llvm.org >>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>> >>> >>> >> > <build_llvm-toolchain_clang-cmake-ninja.sh><install_llvm-toolchain_clang-cmake-ninja.sh>