Sjoerd Meijer via llvm-dev
2020-Sep-07 12:50 UTC
[llvm-dev] LTO'd / PGO'd Pre-built Release Binary
A compiler built with LTO / PGO can significantly improve build times. Locally I see ~10% reduction in compilation times of Debug or Release versions of clang/llvm using an LTO'd clang compiler. This is not really news of course (e.g. recently also reconfirmed in https://linuxplumbersconf.org/event/7/contributions/802/), but that made me curious if we can and/or should enable LTO and/or PGO to create our prebuilt binaries? The disadvantage for our release people will be obvious: that will take a lot more time and resources. But would it be worth to make this investment? Cheers, Sjoerd. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200907/40da8714/attachment.html>
Tobias Hieta via llvm-dev
2020-Sep-07 13:56 UTC
[llvm-dev] LTO'd / PGO'd Pre-built Release Binary
Sjoerd, I emailed llvm-dev a while ago suggesting that we at least turn on LTO. I really didn't get much feedback on that unfortunately, I think most people get clang via their distro or compile it themselves. I did look into what's needed: the release script need to build the first and second stage as it does now but for the final stage we would need to adapt it to send in a compatible linker (gold or lld) and fix ranlib/ar invocations. I think the most complicated part is that the release testers are the one responsible for building the final binaries and there is a lot of different systems that might require some tweaking. I have had a dream of some CI service being able to build PGO/LTO optimized binaries (at least for Linux, windows and Mac) for each release - but I think we would need a cloud vendor sponsoring that CPU time (as a reference our toolchain build takes ~5 hrs for something similar). Thanks, Tobias On Mon, Sep 7, 2020, 14:50 Sjoerd Meijer via llvm-dev < llvm-dev at lists.llvm.org> wrote:> A compiler built with LTO / PGO can significantly improve build times. > Locally I see ~10% reduction in compilation times of Debug or Release > versions of clang/llvm using an LTO'd clang compiler. This is not really > news of course (e.g. recently also reconfirmed in > https://linuxplumbersconf.org/event/7/contributions/802/), but that made > me curious if we can and/or should enable LTO and/or PGO to create our > prebuilt binaries? > > The disadvantage for our release people will be obvious: that will take a > lot more time and resources. But would it be worth to make this investment? > > Cheers, > Sjoerd. > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200907/7de6dd82/attachment-0001.html>
Sjoerd Meijer via llvm-dev
2020-Sep-07 14:10 UTC
[llvm-dev] LTO'd / PGO'd Pre-built Release Binary
> I emailed llvm-dev a while ago suggesting that we at least turn on LTO.Ah, sorry, missed that. Otherwise I would have replied to/revived that thread.> I really didn't get much feedback on that unfortunately, I think most people get clang via their distro or compile it themselves.Well, that's what I was wondering, wasn't sure if it is worth doing....but I guess we upload these binaries for a reason and guess these binaries are picked up by some people? Even if not, it would be good to set an example (and for testing)?> I think the most complicated part is that the release testers are the one responsible for building the final binaries and there is a lot of different systems that might require some tweaking.Oh yeah, definitely. I see that it's easy to suggest this, and that it's a lot of work for the people who have to do this. Cheers. ________________________________ From: Tobias Hieta <tobias at plexapp.com> Sent: 07 September 2020 14:56 To: Sjoerd Meijer <Sjoerd.Meijer at arm.com> Cc: llvm-dev <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] LTO'd / PGO'd Pre-built Release Binary Sjoerd, I emailed llvm-dev a while ago suggesting that we at least turn on LTO. I really didn't get much feedback on that unfortunately, I think most people get clang via their distro or compile it themselves. I did look into what's needed: the release script need to build the first and second stage as it does now but for the final stage we would need to adapt it to send in a compatible linker (gold or lld) and fix ranlib/ar invocations. I think the most complicated part is that the release testers are the one responsible for building the final binaries and there is a lot of different systems that might require some tweaking. I have had a dream of some CI service being able to build PGO/LTO optimized binaries (at least for Linux, windows and Mac) for each release - but I think we would need a cloud vendor sponsoring that CPU time (as a reference our toolchain build takes ~5 hrs for something similar). Thanks, Tobias On Mon, Sep 7, 2020, 14:50 Sjoerd Meijer via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: A compiler built with LTO / PGO can significantly improve build times. Locally I see ~10% reduction in compilation times of Debug or Release versions of clang/llvm using an LTO'd clang compiler. This is not really news of course (e.g. recently also reconfirmed in https://linuxplumbersconf.org/event/7/contributions/802/), but that made me curious if we can and/or should enable LTO and/or PGO to create our prebuilt binaries? The disadvantage for our release people will be obvious: that will take a lot more time and resources. But would it be worth to make this investment? Cheers, Sjoerd. _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200907/d0849ff0/attachment.html>