llvm dev - Apr 2021 - Incrementally compiling LLVM

On 14/04/2021 21:18, David Blaikie wrote:
> Sounds like something's going wrong for sure. How are you measuring
> the time? (how much time is it taking) and what are you testing by
> "partial recompilation"? Touching an ADT .h file is likely to
> recompile nearly everything so might not be much better than a clean
> build - but touching a .cpp file (especially a .cpp file for a single
> tool, rather than a library) might be quite quick. So try timing a
> full clean build (ninja -t clean, ninja clang) and then incrementally
> touching just a .cpp file (eg: clang/tools/driver/driver.cpp  ) and
> see how they compare
I currently do not have sufficient resources to compile LLVM on my
personal machine, so I am (ab?)using GitHub Actions. My workflow is
setup such that the build artifacts are cached and restored on every
worflow run. The time is measured automatically by GitHub itself, and is
around ~2h for each run, whether from scratch or starting from the cache.

I don't generally edit .h files directly, unless they are generated by
modifying TableGen files.

The fact that someone else has successfully been able to do incremental
builds _does_ make it seem like it's a configuration issue on my part.

On Wed, Apr 14, 2021 at 12:59 PM Francesco Bertolaccini via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 14/04/2021 21:18, David Blaikie wrote:
> > Sounds like something's going wrong for sure. How are you
measuring
> > the time? (how much time is it taking) and what are you testing by
> > "partial recompilation"? Touching an ADT .h file is likely
to
> > recompile nearly everything so might not be much better than a clean
> > build - but touching a .cpp file (especially a .cpp file for a single
> > tool, rather than a library) might be quite quick. So try timing a
> > full clean build (ninja -t clean, ninja clang) and then incrementally
> > touching just a .cpp file (eg: clang/tools/driver/driver.cpp  ) and
> > see how they compare
>
> I currently do not have sufficient resources to compile LLVM on my
> personal machine, so I am (ab?)using GitHub Actions. My workflow is
> setup such that the build artifacts are cached and restored on every
> worflow run. The time is measured automatically by GitHub itself, and is
> around ~2h for each run, whether from scratch or starting from the cache.
>
For doing incremental compilation in such a context, I'd try to enable some
sort of CCACHE instead, we have this on a bot and it's quite effective in
this context!

It's not using GitHub actions thought, but here is a recipe for GitHub
actions (I haven't tried it):
https://cristianadam.eu/20200113/speeding-up-c-plus-plus-github-actions-using-ccache/

Best,

-- 
Mehdi

> I don't generally edit .h files directly, unless they are generated by
> modifying TableGen files.
>
> The fact that someone else has successfully been able to do incremental
> builds _does_ make it seem like it's a configuration issue on my part.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210414/5093c3a2/attachment.html>

On Wed, Apr 14, 2021 at 12:58 PM Francesco Bertolaccini
<francesco at bertolaccini.dev> wrote:
>
> On 14/04/2021 21:18, David Blaikie wrote:
> > Sounds like something's going wrong for sure. How are you
measuring
> > the time? (how much time is it taking) and what are you testing by
> > "partial recompilation"? Touching an ADT .h file is likely
to
> > recompile nearly everything so might not be much better than a clean
> > build - but touching a .cpp file (especially a .cpp file for a single
> > tool, rather than a library) might be quite quick. So try timing a
> > full clean build (ninja -t clean, ninja clang) and then incrementally
> > touching just a .cpp file (eg: clang/tools/driver/driver.cpp  ) and
> > see how they compare
>
> I currently do not have sufficient resources to compile LLVM on my
> personal machine,
A side note, have you tried these things:
https://llvm.org/docs/GettingStarted.html#common-problems - often use
of binutils ld makes it difficult to build/link (especially with high
parallelism of multicore processors) & switching to gold or lld can
make things much more manageable.
> so I am (ab?)using GitHub Actions. My workflow is
> setup such that the build artifacts are cached and restored on every
> worflow run. The time is measured automatically by GitHub itself, and is
> around ~2h for each run, whether from scratch or starting from the cache.
>
> I don't generally edit .h files directly, unless they are generated by
> modifying TableGen files.
>
> The fact that someone else has successfully been able to do incremental
> builds _does_ make it seem like it's a configuration issue on my part.
>

This flow is being used successfully by both npcomp and circt, which are
llvm incubator projects.  You should be able to leverage/build off of the
github actions flows that they define.  see https://github.com/llvm/circt
and https://github.com/llvm/mlir-npcomp

Steve

On Wed, Apr 14, 2021 at 12:59 PM Francesco Bertolaccini via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On 14/04/2021 21:18, David Blaikie wrote:
> > Sounds like something's going wrong for sure. How are you
measuring
> > the time? (how much time is it taking) and what are you testing by
> > "partial recompilation"? Touching an ADT .h file is likely
to
> > recompile nearly everything so might not be much better than a clean
> > build - but touching a .cpp file (especially a .cpp file for a single
> > tool, rather than a library) might be quite quick. So try timing a
> > full clean build (ninja -t clean, ninja clang) and then incrementally
> > touching just a .cpp file (eg: clang/tools/driver/driver.cpp  ) and
> > see how they compare
>
> I currently do not have sufficient resources to compile LLVM on my
> personal machine, so I am (ab?)using GitHub Actions. My workflow is
> setup such that the build artifacts are cached and restored on every
> worflow run. The time is measured automatically by GitHub itself, and is
> around ~2h for each run, whether from scratch or starting from the cache.
>
> I don't generally edit .h files directly, unless they are generated by
> modifying TableGen files.
>
> The fact that someone else has successfully been able to do incremental
> builds _does_ make it seem like it's a configuration issue on my part.
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20210414/e0edd5b7/attachment.html>