Chris Bieneman via llvm-dev
2016-Feb-10 18:01 UTC
[llvm-dev] [FYI] CMake's Ninja generator is non-deterministic
It is with great sadness that I must tell everyone CMake’s Ninja generator is non-deterministic (https://public.kitware.com/Bug/view.php?id=15968 <https://public.kitware.com/Bug/view.php?id=15968>). I’m not sure if this impacts all versions of CMake, but it certainly impacts all the recent releases. You might ask why this matters? Sadly the non-determinism *does* impact determinism in the final builds. I haven’t fully dug into the extent of the differences, but I was seeing non-reproducibility in LTO builds. Whatever Ninja is doing differently causes LTO to make different optimization decisions. I have verified that this does not impact CMake’s makefile generator. The important takeaway from this is that if you are building for a purpose where non-determinism can impact your results, don’t use Ninja. For most engineering tasks building and testing locally Ninja, as always, works great. For incremental bots testing correctness, no problem. Any bots testing performance need to move to a deterministic generator (i.e. Unix Makefiles). Anyone building or testing a release of LLVM or clang for packaging purposes should also use a deterministic generator. Sorry to be the bearer of bad news… -Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160210/43c0a7fb/attachment.html>
Mehdi Amini via llvm-dev
2016-Feb-10 22:12 UTC
[llvm-dev] [FYI] CMake's Ninja generator is non-deterministic
From what I see reproducing offline, it is the order of dependencies that is changing (I bet they iterate over an unordered container): Example: < build cmake_order_depends_target_llvm-profdata: phony || lib/libLLVMSupport.a lib/libLLVMCore.a lib/libLLVMBitReader.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObject.a lib/libLLVMProfileData.a ---> build cmake_order_depends_target_llvm-profdata: phony || lib/libLLVMProfileData.a lib/libLLVMSupport.a lib/libLLVMCore.a lib/libLLVMBitReader.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObject.aThe consequence is the order of the libraries is not the same on the command line. Have you seen other differences? You may want to update the bug report with these information if you think it could be useful. -- Mehdi> On Feb 10, 2016, at 10:01 AM, Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > It is with great sadness that I must tell everyone CMake’s Ninja generator is non-deterministic (https://public.kitware.com/Bug/view.php?id=15968 <https://public.kitware.com/Bug/view.php?id=15968>). > > I’m not sure if this impacts all versions of CMake, but it certainly impacts all the recent releases. You might ask why this matters? Sadly the non-determinism *does* impact determinism in the final builds. I haven’t fully dug into the extent of the differences, but I was seeing non-reproducibility in LTO builds. Whatever Ninja is doing differently causes LTO to make different optimization decisions. I have verified that this does not impact CMake’s makefile generator. > > The important takeaway from this is that if you are building for a purpose where non-determinism can impact your results, don’t use Ninja. For most engineering tasks building and testing locally Ninja, as always, works great. For incremental bots testing correctness, no problem. > > Any bots testing performance need to move to a deterministic generator (i.e. Unix Makefiles). Anyone building or testing a release of LLVM or clang for packaging purposes should also use a deterministic generator. > > Sorry to be the bearer of bad news… > > -Chris > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160210/bb0b679d/attachment.html>
Chris Bieneman via llvm-dev
2016-Feb-10 22:14 UTC
[llvm-dev] [FYI] CMake's Ninja generator is non-deterministic
Brad King has already replied with a patch that removes many of the differences (but not all). It was iterating over a std::map keyed on pointers. -Chris> On Feb 10, 2016, at 2:12 PM, Mehdi Amini <mehdi.amini at apple.com> wrote: > > From what I see reproducing offline, it is the order of dependencies that is changing (I bet they iterate over an unordered container): > > Example: > > < build cmake_order_depends_target_llvm-profdata: phony || lib/libLLVMSupport.a lib/libLLVMCore.a lib/libLLVMBitReader.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObject.a lib/libLLVMProfileData.a > --- > > build cmake_order_depends_target_llvm-profdata: phony || lib/libLLVMProfileData.a lib/libLLVMSupport.a lib/libLLVMCore.a lib/libLLVMBitReader.a lib/libLLVMMC.a lib/libLLVMMCParser.a lib/libLLVMObject.a > > > The consequence is the order of the libraries is not the same on the command line. > Have you seen other differences? > > You may want to update the bug report with these information if you think it could be useful. > > > -- > Mehdi > > > >> On Feb 10, 2016, at 10:01 AM, Chris Bieneman via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: >> >> It is with great sadness that I must tell everyone CMake’s Ninja generator is non-deterministic (https://public.kitware.com/Bug/view.php?id=15968 <https://public.kitware.com/Bug/view.php?id=15968>). >> >> I’m not sure if this impacts all versions of CMake, but it certainly impacts all the recent releases. You might ask why this matters? Sadly the non-determinism *does* impact determinism in the final builds. I haven’t fully dug into the extent of the differences, but I was seeing non-reproducibility in LTO builds. Whatever Ninja is doing differently causes LTO to make different optimization decisions. I have verified that this does not impact CMake’s makefile generator. >> >> The important takeaway from this is that if you are building for a purpose where non-determinism can impact your results, don’t use Ninja. For most engineering tasks building and testing locally Ninja, as always, works great. For incremental bots testing correctness, no problem. >> >> Any bots testing performance need to move to a deterministic generator (i.e. Unix Makefiles). Anyone building or testing a release of LLVM or clang for packaging purposes should also use a deterministic generator. >> >> Sorry to be the bearer of bad news… >> >> -Chris >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160210/5692f48b/attachment.html>